算法时代

The Algorithmic Age

我们喜欢给时间段贴上标签，也许是因为给时间贴上标签可以让我们掌握它的流动性。因此，我们开始将现在称为新算法时代的曙光，在这个时代，算法将占据主导地位，并将控制我们生活中越来越大的部分。有趣的是，我们不再谈论计算机时代或互联网时代。我们不知何故将它们视为理所当然。当我们添加算法时，我们开始暗示也许一些本质上不同的事情已经开始发生。“看看全能的算法，一段计算机代码在我们这个世俗时代代表着更高的权威，一种神，”前《纽约时报》记者、 ​​Radio Open Source节目主持人克里斯托弗·莱登说。的确，当算法被用来组织政治运动、追踪我们在网络世界的踪迹、跟踪我们的购物行为并精准投放广告、推荐约会对象或监测我们的健康状况时，它们就被视为某种形式的更高权威。1

We like putting labels on time periods, perhaps because affixing a tab on time allows us to get a grip on its fluidity. We have therefore started speaking of the present as the dawning of a new algorithmic age, in which algorithms will reign supreme, and will govern larger and larger parts of our lives. It is interesting that we are not talking about the computer age or internet age anymore. We somehow take them for granted. It is when we add algorithms that we begin intimating that perhaps something qualitatively different has started taking place. “Behold the Almighty Algorithm, a snippet of computer code coming to stand for a Higher Authority in our secular age, a sort of god,” says Christopher Lydon, former New York Times journalist and host of the Radio Open Source show. And indeed, algorithms are taken to be some form of higher authority when they are used to organize political campaigns, follow our traces in the online realm, shadow our shopping and target us with advertising, suggest dating partners, or monitor our health.1

这一切都笼罩着一层神秘的氛围，或许这反而让算法的追随者们沾沾自喜。被描述为“程序员”或“计算机科学家”，标志着你是一个正派的人，尽管略带技术性。成为这个即将改变我们生活几乎所有事物的群体的一员，岂不更令人欣慰？

There is an aura of mystery around all that, which perhaps flatters the acolytes of algorithms. Being described a “programmer” or “computer scientist” marks you as a decent, albeit somewhat technical, character. How much better to be a member of the tribe that is about to change almost everything in our lives?

算法确实在某种程度上被视为某种神明。它们大多不负责任，就像神明一样；事情的发生并非出于人类的意志，而是由算法决定的，而算法本身却不承担责任。运行算法的机器能够在越来越多的领域超越人类，以至于人类的优势似乎正在日益缩小；有些人认为，计算机在认知的各个方面超越人类的那一天并不遥远。

There is definitely a sense in which algorithms are a sort of god. They are mostly held unaccountable, like gods; things happen, not because of human agency, but because they were decided by an algorithm, and the algorithm sits beyond the pale of responsibility. Machines, running algorithms, can surpass human performance in more and more fields so that it appears that the area of human superiority is reduced day by day; some believe that the day where computers will be able to surpass humans in every aspect of cognition is not far away.

但也有某种意义，算法并不像神，尽管我们常常忽略这一点。算法并非通过启示而产生其结果。我们确切地知道它遵循的规则和它采取的步骤。无论结果多么精彩，它总能追溯到一些基本的运算。对于算法新手来说，这些运算如此简单，或许会让他们感到惊讶。这并不是要贬低算法；了解某个事物真正的运作方式，或许能揭开它神秘面纱的一部分。同时，理解某个事物的运作方式，或许能让我们欣赏其设计的优雅，即使它不再神秘。

But there is also a sense in which algorithms are nothing like gods, although we often lose sight of it. An algorithm does not produce its results by an act of revelation. We know exactly the rules that it follows and kinds of steps it takes. No matter how wonderful the outcome, it can always be traced back to some elementary operations. To people who are newcomers to algorithms, it may come as a surprise how elementary these may be. That is not to besmirch algorithms; seeing how something really works may take out some part of its mystique. At the same time, understanding how something works may allow us to appreciate the elegance of its design, even if it is no longer mysterious.

本书的前提是，算法其实并不神秘。它们是让我们能够出色完成某些任务的工具；它们是特定类型的工具，其目的是帮助我们解决问题。从这个意义上讲，它们是认知工具；因此，它们并非唯一的工具。数字和算术也是认知工具。我们花了数千年的时间才发展出一套孩子们可以在学校学习的数字系统，使他们能够进行一些没有数字系统就无法进行的计算。如今，我们认为计算能力是理所当然的，但在几代人之前，只有极少数人了解它。

The premise of this book is that indeed algorithms are not mysterious. They are tools that allow us to do certain things well; they are specific kinds of tools whose purpose is to allow us to solve problems. In this way they are cognitive tools; as such, they are not the only ones. Numbers and arithmetic are also cognitive tools. It took us thousands of years to evolve a number system that children can learn in school so that they can perform calculations that would be impossible without it. Now we take numeracy for granted, but a few generations back only a small minority of humans had any knowledge of it.

同样，算法知识不应成为少数精英的特权；作为认知工具，它们可以被各种各样的人理解，而不仅仅是计算机专业人士。更重要的是，它们应该被更多人理解，因为这将使我们能够正确看待算法：了解它们做什么、它们如何做，以及我们可以实际地期望它们做什么。

Similarly, knowledge of algorithms should not be the prerogative of a small elite minority; as cognitive tools they can be apprehended by all kinds of people, not just computer professionals. What is more, they should be understood by more people because that will allow us to put algorithms into perspective: to know what they do, how they do it, and what we can realistically expect them to do.

我们追求的是算法的基本知识，这样我们才能在算法时代的讨论中发挥有意义的作用。这个时代并非强加于我们，而是我们基于自己设计的工具自主创造的时代。本书的主题正是对这些工具的研究。算法是优美的工具，了解它们的构造和工作原理可以提升我们的思维方式。

An essential knowledge of algorithms is what we are after here so that we can take a meaningful part in the conversations on the algorithmic age. That is not an age that is thrust on us, but one of our own creation, based on tools that we have devised. The study of these tools is the subject of this book. Algorithms are beautiful tools, and a glimpse of how they are made and work can enhance our way of thinking.

我们先来澄清一个令人厌烦的观念：算法是计算机的专利。我们会发现，这和说数字是计算器的专利一样有道理。

We’ll start by dispelling an irksome notion: that algorithms are about computers. This, we’ll see, makes as much sense as saying that numbers are about calculators.

做事的方法

A Way to Do Things

纸笔谜题、音乐、数的可除性以及粒子物理学中的中子加速器——我们会发现，它们的共同点在于相同的算法，应用于如此不同的领域，却基于相同的底层原理。这怎么可能呢？

A pen-and-paper puzzle, music, divisibility of numbers, and neutron accelerators in particle physics—we’ll see that what they all have in common is the same algorithm, applied to such different domains, yet working on the same underlying principles. How can this be?

“算法”一词本身并不能揭示其含义。它源于穆罕默德·伊本·穆萨·花拉子密（约780年-约850年）的名字，他是一位波斯学者，研究数学、天文学和地理学。花拉子密的贡献广泛而广泛。“代数”一词源于他最具影响力的著作《计算简明书》的阿拉伯语书名。 完成和平衡。他的第二本最具影响力的著作是《印度数字计算法》，该书论述了算术，后被译成拉丁文，将印度-阿拉伯数字系统引入西方。花剌子密的名字拉丁化为Algorismus，意为使用十进制数进行数值计算的方法。Algorismus 受希腊语中“数字”一词（arithmos，即算术）的影响，后来演变为 algorithm，仍指十进制算术，直到 19 世纪才获得其现代含义。

The word “algorithm” itself does not reveal its meaning. It comes from the name of Muḥammad ibn Mūsā al-Khwārizmī (ca. 780–ca. 850), a Persian scholar who worked on mathematics, astronomy, and geography. Al-Khwārizmī’s contributions were many and widespread. The term “algebra” comes from the Arabic title of his most influential work, The Compendious Book on Calculation by Completion and Balancing. His second most influential book, On the Calculation with Hindu Numerals, was on arithmetic and, translated into Latin, introduced the Hindu-Arabic numeral system to the West. Al-Khwārizmī’s name was latinized to Algorismus, which came to denote the method of numerical computation with the decimal numbers. Algorismus, influenced by the Greek word for “number” (arithmos, as in arithmetic), became algorithm, still denoting decimal arithmetic, before acquiring its modern sense in the nineteenth century.

你可能会认为算法是我们用计算机完成的事情，但这是错误的。这是错误的，因为我们在计算机出现之前很久就有了算法。已知的第一个算法可以追溯到古巴比伦。2这也是一个错误，因为算法与计算机无关。算法是关于以特定方式做某事，遵循某些步骤。这有点模糊。你可能会问，什么样的步骤？什么具体的方式？我们可以消除所有模糊性，并给出算法是什么以及它做什么的精确数学定义——这样的定义确实存在——但我们不需要走这么远。你可能会很高兴地知道，算法是一组可以用笔和纸遵循的步骤，你可以放心，这个看似简单的描述与数学家和计算机科学家使用的描述很接近。

You could be tempted to think that algorithms are something that we do with computers, but this would be wrong. It is wrong because we had algorithms long before we had computers. The first-known algorithms date back to ancient Babylon.2 It is also wrong because algorithms are not about problems that have to do with computers. Algorithms are about doing something in a specific way, following some kind of steps. That is somewhat vague. You may ask, What kind of steps? What specific way? We can dismiss all vagueness, and give a precise mathematical definition of what an algorithm is and what it does—such a definition does exist—but we don’t need to go to such lengths. You may be happy to know that an algorithm is a set of steps that you can follow with pen and paper, and you can be assured that this seemingly facile description is close to those used by mathematicians and computer scientists.

因此，我们可以从一个只需写下来就能解决的问题开始我们的算法探索。假设我们有两组对象，并且希望将其中一组的对象尽可能均匀地分布在另一组的对象中。我们将对第一组的对象使用叉号 ( )，对第二组的对象使用项目符号 ( )。我们希望将叉号分布在项目符号之间。

So we can start our approach to algorithms with a problem that we can solve by just writing things down. Suppose we have two sets of objects and want to spread the objects of one of the two sets as evenly as possible among the objects of the other set. We will use crosses () for the objects of the first set and bullets () for the objects of the second set. We want to spread out crosses among the bullets.

如果叉号的数量能整除物体总数，那就很容易了。我们只需像做除法一样，把叉号分成几份，再分成几份。例如，如果我们总共有 12 个物体，其中 3 个是叉号，9 个是子弹号，那么我们先放一个叉号，然后放三个子弹号，再放一个叉号，三个子弹号，最后再放一个叉号，三个子弹号：

If the number of crosses divides the total number of objects, that is easy. We just partition the crosses among the bullets as if we would do division. For example, if we have 12 objects in total, out of which three are crosses and nine are bullets, we put one cross, then three bullets, then one cross, three bullets, and finally another cross and three bullets:

但是，如果所有物体（十字和子弹）的总数不能被十字的数量整除怎么办？如果我们有五个十字和七颗子弹，该怎么办？

But what if the total number of objects, crosses and bullets taken together, cannot be divided exactly by the crosses? What can we do if we have five crosses and seven bullets?

我们首先将所有叉号和所有项目符号放在一行中，如下所示：

We start by putting all crosses followed by all bullets in one row as follows:

然后我们取出五颗子弹并将它们放在十字架下面：

Then we take five bullets and place them under the crosses:

我们注意到，在出现的模式中，右边有两列余数。我们将这两列余数（每列包含一个项目符号）放在前两列下方，形成第三行：

We notice in the pattern that emerges that we have a remainder of two columns to the right. We take the two remainder columns, each comprising a single bullet, and put them under the first two columns, forming a third row:

现在我们注意到还有三列余数。我们取最右边的两列，并把它们放在最左边的两列下面：

Now we notice that we have a remainder of three columns. We take the rightmost two of them and put them under the two leftmost columns:

现在我们只剩下一列余数，所以我们停止。我们将这些列从左到右连接起来，得到：

Now we have only one remainder column, so we stop. We concatenate the columns from left to right and get:

这就是结果。我们把十字标记分布在了项目符号中。它们的间距不像以前那么均匀，但这是不可能的，因为记住，5不能整除12。不过，我们设法避免了把所有十字标记都堆在一起，并且创建了一个看起来并非完全杂乱无章的图案。

This is the result. We have distributed the crosses among the bullets. They are not as evenly spaced as before, but that is impossible to do because, remember, five does not divide evenly into 12. We have managed to avoid heaping all the crosses together, however, and have created a pattern that does not look entirely haphazard.

您可能想知道这种模式是否有什么特别之处；如果您用 DUM 代替十字，用 da 代替子弹，会更有帮助。然后模式变成 DUM-da-da-DUM-da-DUM-da-da-DUM-da-DUM-da，它真的是一种节奏。节奏由重音部分（也称为起始符）和非重音或静音部分组成。我们发现的节奏不是我们自己设计的节奏。中非共和国的阿卡俾格米人使用它；它是南非歌曲中称为 Venda 的拍手声；它也是巴尔干半岛马其顿使用的一种节奏模式。还有更多。如果我们旋转它，使它从第二个十字（即起始符）开始，那么它就变成：

You may wonder if there is anything particular about this pattern; it helps if you substitute DUM for the cross and da for the bullet. Then the pattern goes DUM-da-da-DUM-da-DUM-da-da-DUM-da-DUM-da and it really is a rhythm. A rhythm is constituted by accented parts, also called onsets, and unaccented or silent parts. The rhythm we found is not a rhythm of our own devising. It is used by the Aka pygmies in the Central African Republic; it is the clapping, called Venda, of a South African song; it is also a rhythm pattern used in Macedonia, in the Balkans. There is more. If we rotate it, so that it starts at the second cross (that is, onset), then it becomes:

这是哥伦比亚钟声模式，在古巴和西非很流行，也是肯尼亚的一种鼓声模式，在马其顿也有使用（再次）。如果我们把它旋转到从第三、第四和第五个音开始，世界各地其他流行的节奏就出现了。

That is the Columbia bell pattern, popular in Cuba and West Africa, as well as a drumming pattern in Kenya, while it is also used in Macedonia (again). If we rotate it to start on the third, fourth, and fifth onset, other popular rhythms around the world emerge.

这仅仅是一次性的事情吗？我们可以尝试用七个起音和五个静默部分创建一个12部分的节奏——有点像我们之前提到的五个起音和七个静默部分的镜像。如果我们遵循完全相同的步骤，我们将得到：

Is this just a one-off thing? We can try to create a 12-part rhythm out of seven onsets and five silent parts—kind of mirroring the five onsets and seven silent parts that we had before. If we follow exactly the same procedure, we will arrive at:

这又是一个节奏。加纳阿散蒂人的Mpre节奏中也用到了它。如果我们从最后一个声母开始，尼日利亚的约鲁巴人、中非人和塞拉利昂人也都用它。

This, again, is a rhythm. It is used in the Mpre rhythm of the Ashanti in Ghana, and if we start it on the last onset, it is used by the Yoruba in Nigeria as well as in Central Africa and Sierra Leone.

以免您认为我们有地理遗漏，如果我们从 5 个节拍和 11 个静音部分开始，我们会得出以下结论：

Lest you think we have geographic omissions, if we start with five beats and 11 silent parts, we arrive at the following:

这就是旋转的Bossa-Nova节奏。真正的Bossa-Nova节奏从第三个音符开始，所以确切的对应关系是：

That is the Bossa-Nova rhythm, rotated. The actual Bossa-Nova rhythm starts on the third onset, so the exact correspondence is:

如果我们尝试用三个节拍和四个静默部分，我们会得到以下模式：

If we try with three beats and four silent parts, we get the pattern:

这种七四拍子的节奏很流行，而且不仅仅出现在传统音乐中。在其他曲调中，Pink Floyd 的歌曲《Money》也采用了这种节奏模式：

This rhythm in a seven/four meter is popular, and not just in traditional music. Among other tunes, it is the rhythmic pattern of Pink Floyd’s song “Money”:

通过在列中放置十字和项目符号，并按照我们刚才描述的方式移动它们，可以衍生出更多的节奏。我们通过测量余数列来演示这个过程，但这实际上是一种图形化的方式，用来展示实际情况。我们不需要创建列、检查几何形状并移动它们，而是可以用简单的数值运算更正式地完成同样的操作。为了看清楚，让我们回到 12 个声部和 7 个声部的例子。我们首先用 12 除以 7，得到商 1 和余数 5：

Many more rhythms can be derived in this way by putting crosses and bullets in columns, and moving them around in the way we just described. We illustrated the procedure by measuring remainder columns, but this really is a pictorial way of showing what really happens. Instead of creating columns, checking the geometry, and moving them around, we can do the same thing more formally with simple numerical operations. To see what, let’s return to the example of 12 parts and seven onsets. We start by dividing 12 by 7, which gives us quotient 1 and remainder 5:

这告诉我们将七个声母放在开头，形成七列声母，然后是其余五个非重音部分：

This tells us to put the seven onsets in the beginning, creating seven columns of onsets, followed by a remainder of the five unaccented parts:

现在我们再次进行除法，但这次我们用前一次除法的除数 7 除以前一次除法的余数 5。这再次得出商为 1，而余数为 2：

Now we divide again, but this time we divide the divisor of the previous division, 7, by the remainder of the previous division, 5. This gives us a quotient of 1 again while the remainder is 2:

这意味着我们需要取出最右边的五列并将它们放在最左边的五列下面，余数为 2：

This means that we need to take the five rightmost columns and place them under the five leftmost columns, leaving a remainder of 2:

我们重复同样的步骤：用前一个除法的除数 5 除以上一个除法的余数 2。商为 2，余数为 1：

We repeat the same step: we divide the divisor of the previous division, 5, by the remainder of the previous division, 2. The quotient is 2 and the remainder is 1:

这告诉我们取最右边两列的两倍并将它们放在最左边两列的下面，余数为 1：

This tells us to take twice the two rightmost columns and place them under the two leftmost columns, leaving a remainder of 1:

注意，这里的“两倍”意味着如果我们按照之前的方式操作，不用除法，那么这相当于我们分两步完成的操作。我们可以这样操作：

Note that twice means that this is equivalent to what we would be doing in two steps if we had worked as we were doing before, without using the division. We would go from:

首先：

first to:

然后：

and then to:

如果我们连接这些列，我们就会得到 Mpre 节奏：

If we concatenate the columns, we get the Mpre rhythm:

我们的第一个算法

Our First Algorithm

我们可以用更精确的术语将我们遵循的方法写成以下步骤。假设我们从两个数字a和b开始。令a为音节的总数。如果起始音的数量大于静音音节的数量，则b 为起始音的数量。否则， b为静音音节的数量。首先，我们创建一行，其中起始音后接静音音节。

We can write down the method we followed in a bit more precise terms as the following steps. We assume that we start with two numbers, a and b. We let a be the total number of parts. If the number of onsets is greater than the number of the silent parts, then b is the number of onsets. Otherwise it is the number of the silent parts. At the beginning, we create a row with the onsets followed by the silent parts.

1. 1. 将a除以b。这将得到商和余数。如果我们将商记为q，将余数记为r，则得到。这就是我们所知的整数除法。我们取q乘以最右边的b列并将它们移动到最左边的列下，其余的r列留在右边。
2. 1. Perform the division of a by b. This will give us a quotient and remainder. If we call the quotient q and remainder r, we’ll have . This is integer division as we know it. We take q times the rightmost b columns and move them under the leftmost columns, leaving a remainder of r columns on the right.
3. 2. 如果余数r等于零或一，则停止。否则，我们回到步骤 1，但这次b将成为新的a，r将成为新的b。或者换句话说，我们回到步骤 1，将a设置为b，将b设置为r。
4. 2. If the remainder r is equal to zero or one, then we stop. Otherwise, we go back to step 1, but this time b will be the new a and r will be the new b. Or in other words, we go back to step 1, setting a equal to b and b equal to r.

在这两个步骤中，我们重复执行除法，直到重复执行没有意义为止。您可以在下表中追踪我们执行的步骤，其中我们从 和 开始，就像我们之前所做的那样；在每一行中，我们都有：

In these two steps we perform a division repeatedly, until it does not make sense to repeat it. You can trace the steps we take in the following table, where we start with and , like we did before; in each row we have :

如果你检查表格，就能验证每一行对应着列的形成和移动的一个步骤，但我们对所使用的方法有更精确的定义。事实上，我们有一系列可以用纸笔执行的步骤，所以这就是我们的第一个算法！我们有一个算法，可以创建与许多（实际上数量惊人的）音乐节奏相对应的模式。使用不同数量的偏移量加上无声部分，我们可以得到大约40种节奏模式，这些模式在世界各地的不同节奏中都能找到。这应该让我们停下来思考一下：这是一个简单的算法（只有两步，重复），却能产生如此多有趣的结果。

If you examine the table, you can verify that each row corresponds to one step of the column formation and moving, but we have a more precise definition of the method we used. In fact, we have a series of steps that we can perform with pen and paper, so this is our first algorithm! We have an algorithm for creating patterns that correspond to many, and indeed surprisingly many, musical rhythms. Working with different numbers of offsets and silent parts, we can get about 40 rhythmic patterns that are found in different rhythms around the world. That should give us pause for a minute: it is a simple algorithm (only two steps, repeated) and yet able to produce so many interesting results.

但是，我们的算法能做的不止这些。既然我们正在讨论两个数字的除法，让我们考虑以下一般问题：如果我们有两个数字a和b，那么能整除它们的最大数字是多少？这称为这两个数字的最大公约数，或gcd。我们在初等算术中会遇到最大公约数，例如，如果我们有 12 包饼干和 4 包奶酪，你如何将它们分配到篮子里，以便每个篮子中的饼干和奶酪的比例相同？因为四能整除 12，所以你会有四个篮子，每个篮子包含三包饼干和一包奶酪；12 和四的最大公约数是四。如果你有 12 包饼干和八包奶酪，事情会变得更有趣。你不能用一个数除以另一个数，但是能同时整除 12 和 8 的最大数是四，这意味着你将再次制作四个篮子，每个篮子包含三包饼干和两包奶酪。

Our algorithm does more than that, though. As we are talking about the division of two numbers, let us consider the following general problem: If we have two numbers a and b, what is the greatest number that divides them both? This is called the greatest common divisor, or gcd, of the two numbers. We encounter the greatest common divisor in elementary arithmetic, in problems such as, If we have 12 packets of crackers and four packets of cheese, how will you distribute them in baskets so that you have the same proportion of crackers and cheese in each basket? As four divides 12, you will have four baskets, each containing three packets of crackers and one packet of cheese; the greatest common divisor of 12 and four is four. Things get more interesting if you have 12 packets of crackers and eight packets of cheese. You cannot divide one by the other, but the greatest number that divides both 12 and eight is four, which means that you will make again four baskets, each containing three packets of crackers and two packets of cheese.

那么，如何找到任意两个整数的最大公约数呢？我们已经知道，如果其中一个数能整除另一个数，那么这个数就是最大公约数。但如果这种情况没有发生，那么事实证明，为了找到两个数的最大公约数，我们只需要找到两个数相除余数与第二个数的最大公约数。用符号实际上更容易理解。如果我们有两个整数a和b ，那么a和b的最大公约数等于 a和b余数的最大公约数。这又回到了我们的“韵律”上。我们一直以来寻找“韵律”的方法实际上与我们用来寻找两个数之间最大公约数的方法相同。

So how can we find the greatest common divisor of any two integer numbers? We have seen that if one of the numbers divides the other, that is the greatest common divisor. But if that does not happen, then it turns out that in order to find the greatest common divisor of two numbers, we only need to find the greatest common divisor of the remainder of the division of the two numbers and the second number. This is actually easier to see with symbols. If we have two integers a and b, the greatest common divisor of a and b is equal to the greatest common divisor of the remainder of and b. This brings us back to our rhythms. The way we have been finding rhythms is in fact the same way we use to find the greatest common divisor between two numbers.

寻找两个数之间最大公约数的方法被称为欧几里得算法，以纪念古希腊数学家欧几里得，他在其著作《几何原本》（约公元前 300 年）中首次描述了这一算法。其基本思想是，如果我们用两个数中较大的数与较小的数之差代替较大的数，则两个数之间的最大公约数保持不变。以 56 和 24 为例，它们的最大公约数是 8，这也是56 和 24 的最大公约数；32 和 24 也是如此，以此类推。重复减法实际上是除法，因此欧几里得算法的步骤如下：

The way to find the greatest common divisor between two numbers is called Euclid’s algorithm, in honor of Euclid, an ancient Greek mathematician who first described it in his books Elements (ca. 300 BCE). The basic idea is that the greatest common divisor between two numbers remains the same if we replace the larger number of the two with its difference with the smaller number. Take 56 and 24. Their greatest common divisor is 8, which is also the greatest common divisor of and 24, and the same goes for 32 and 24, and so on. Repeated subtraction is really division, so Euclid’s algorithm is described with the following steps:

1. 1. 要求a和b的最大公约数，用a除以b。这样会得到商和余数。如果我们把商记为q，把余数记为r，就会得到。
2. 1. To find the greatest common divisor of a and b, perform the division of a by b. This will give us a quotient and remainder. If we call the quotient q and remainder r, we’ll have .
3. 2. 如果余数r等于 0，则停止， a和b的最大公约数为b。否则，我们回到步骤 1，但这次b将是新的a，r将是新的b。或者换句话说，我们回到步骤 1，将a设置为b，将b设置为r。
4. 2. If the remainder r is equal to 0, then we stop, and the greatest common divisor of a and b is b. Otherwise, we go back to step 1, but this time b will be the new a and r will be the new b. Or in other words, we go back to step 1, setting a equal to b and b equal to r.

这些步骤与之前基本相同。唯一的区别在于，在寻找韵律时，在步骤 2 中，当余数为 0 或 1 时我们停止，而欧几里得算法则在余数为 0 时停止。这实际上是相同的：如果余数为 1，那么在下一次重复步骤 1 时，会得到 0 余数，因为 1 可以整除所有整数。试算 9 和 5：，因此我们转到，然后，因此 9 和 5 的最大公约数是 1。

These are essentially the same steps as before. The only difference is that when finding rhythms, in step 2 we stop when the remainder is 0 or 1, while Euclid’s algorithm stops when the remainder is 0. This is really the same: if you have a remainder of 1, then in the next repetition of step 1, you get a 0 remainder because 1 divides every integer. Try 9 and 5: , so we go to and then , so the greatest common divisor of 9 and 5 is 1.

下表中的和或许能帮助你理解该算法的实际作用，它与我们之前在节奏练习中看到的类似。我们发现 136 和 56 的最大公约数是 8：

It may help you to see the algorithm in action with and in the following table, similar to the one we saw before with our rhythms. We find that the greatest common divisor of 136 and 56 is the number 8:

正如我们在 9 和 5 中提到的那样，欧几里得算法在所有情况下都能正确运行，即使这两个数字不相等除了 1 之外，还有其他公约数。这就是和发生的情况。你可以亲眼看看，如果尝试对和执行算法步骤会发生什么；虽然需要几个步骤，但算法会确定唯一的公约数是 1。

As we noted with 9 and 5, Euclid’s algorithm works correctly in all cases, even when the two numbers do not have any common divisor apart from 1. This is what happened with and . You can see for yourself what happens if you try to perform the algorithm’s steps with and ; it will take a few steps, but the algorithm will determine that the only common divisor is 1.

欧几里得算法的各个步骤按照明确定义的顺序执行。该算法的描述说明了其各个步骤的组合方式：

The steps in Euclid’s algorithm are performed in a well-defined order. The description of the algorithm illustrates the way its component steps are combined:

1. 1. 步骤按顺序排列。
2. 1. The steps are put in a sequence.
3. 2. 步骤可能描述决定后续步骤的选择。在步骤2中，会测试余数是否为0。然后，根据结果，有两种选择：要么停止，要么回到步骤1。
4. 2. Steps may describe a selection that determines which steps to follow. In step 2, there is a test of whether the remainder is 0 or not. Then there are two alternatives, depending on the outcome: we either stop or go back to step 1.
5. 3. 可以将步骤放入循环或迭代中，重复执行。在步骤 2 中，如果余数不等于 0，则返回步骤 1。
6. 3. Steps can be put into a loop or iteration, where they are executed repeatedly. In step 2, if the remainder is not equal to 0, we go back to step 1.

我们将这三种组合步骤的方式称为控制结构，因为它们决定了算法执行时将执行哪些操作。所有算法都以这种方式构建。它们包含执行计算和处理数据的步骤；这些步骤使用这三种控制结构组合在一起并进行编排。更复杂的算法包含更多步骤，并且它们的编排可能更复杂。但这三种控制结构足以描述任何算法的各个步骤应该如何组合在一起。

We call these three ways to combine steps control structures because they dictate which action will be performed as we carry out the algorithm. All algorithms are structured in this way. They comprise steps doing calculations and processing data; these steps are assembled together and choreographed using these three control structures. More complex algorithms have more steps, and their choreography may be more complex. But the three control structures suffice to describe the way the steps of any algorithm should be put together.

算法的各个步骤都会对我们提供的输入进行操作。输入是指算法处理的数据。如果我们采用以数据为中心的观点，我们会使用算法将描述问题的数据转换为与问题解决方案相对应的形式。

The steps of an algorithm will, among other things, operate on the input we provide. The input is the data that are processed by the algorithm. If we adopt a data-centric view, we use an algorithm to transform some data, which describe a problem, to some form that corresponds to the problem’s solution.

我们发现音乐节奏背后有一种算法，它是除法的一种应用，但实际上，我们不必研究那么远；除法本身就是一种算法。即使你没听说过欧几里得算法，你也知道如何除两个大数；我们在早年都花了很多时间学习如何进行长乘法和长除法。我们的老师花了几个小时向我们灌输如何进行这些运算：一套将数字放在正确位置并用它们做事的步骤——它们就是算法。但正如我们刚才看到的，算法不仅仅是关于数字。我们刚刚发现，它们与我们如何创作音乐有关。然而，这并不神秘。节奏是一种在时间间隔内分配压力的方式，当我们包装饼干和奶酪时，同样的原理也在起作用。

We found an algorithm behind musical rhythms that is an application of division, but in reality, we need not look that far; the act of division itself is an algorithm. Even if you have not heard of Euclid’s algorithm, you know how to divide two large numbers; we have all spent time in our early years learning to perform long multiplication and long division. Our teachers spent hours drilling into our heads how to perform these operations: a set of steps for putting numbers in the right places and doing things with them—they are algorithms. But algorithms are not simply about numbers, as we have just seen. We just found that they are about how we can produce music. Yet there is nothing mystifying about that. A rhythm is a way to distribute stresses in a time interval, and the same principle is at work when we pack crackers and cheese.

欧几里得算法在韵律上的应用有一个不太可能的来源：田纳西州橡树岭国家实验室的中子源设施。散裂中子源 (SNS) 产生强脉冲中子束，用于粒子物理实验。（动词“碎裂”的意思是将材料破碎成更小的碎片；在核物理学中，我们用重原子核在被高能粒子轰击后发射出大量质子和中子来描述。）在 SNS 的运行中，一些组件（例如高压电源）应尽可能均匀地运行，以使脉冲在时隙中分布均匀。设计用于分配的算法本质上与节奏算法和欧几里得算法相同，将我们从数字带到亚原子粒子，再到音乐。3

The application of Euclid’s algorithm to rhythms had an unlikely source: a neutron source facility in the Oak Ridge National Laboratory in Tennessee. The Spallation Neutron Source (SNS) there produces intense pulsed neutron beams that are used in experiments in particle physics. (The verb to spall means breaking a material into smaller pieces; in nuclear physics, we have a heavy nucleus emitting a large number of protons and neutrons after being bombarded with a high-energy particle.) In the operation of the SNS, some components, such as high-voltage power supplies, should run so that pulses are distributed in timing slots as evenly as possible. An algorithm devised to do the distribution is essentially the same as the rhythm-making algorithm and Euclid’s algorithm, taking us from numbers to subatomic particles to music.3

算法、计算机和数学

Algorithms, Computers, and Mathematics

我们说过算法与计算机无关，但如今大多数人却将它们捆绑在一起。诚然，算法与计算机结合时才能展现其潜力，但计算机本质上是一种机器，拥有我们可以命令它执行某些操作的特殊能力。我们通过编程来命令它，通常我们编程让它执行算法。

We said that algorithms are not about computers, yet today most people bundle them together. It is true that algorithms show their potential when they are coupled with computers, but a computer is really a machine with the special trait that we can order it to do certain things. We order it by programming it, and usually we program it to execute algorithms.

这引出了编程本身。编程是将我们的意图转化为计算机能够理解的符号的学科。我们称这种符号为编程语言，因为有时看起来我们确实在用人类语言写作，但与人类语言的丰富性和复杂性相比，编程语言是相当简单的。当然，现在计算机实际上什么都不懂。如果我们能够制造出真正智能的机器，未来情况可能会有所改变。但现在，当我们说计算机理解一种符号时，实际上意味着符号被转换成一系列用于操纵电子电路中电流的指令（我们也可能用光代替电流，但原理是一样的）。

Which brings us to programming itself. Programming is the discipline of translating our intentions to some notation that a computer is able to understand. We call this notation a programming language because sometimes it does look like we are writing in a human language, but programming languages are fairly simple affairs compared to the richness and complexity of human languages. Now, of course, a computer does not really understand anything. Things may change in the future, if we are able to produce truly intelligent machines, but right now when we say that a computer understands a notation, it really means that the notation is converted to a series of instructions for manipulating current in electronic circuits (we may also use light instead of electric current, yet the idea is the same).

如果说算法是一组我们可以自己执行的步骤，那么编程就是我们用计算机能理解的符号写下这些步骤的活动。然后由计算机来执行这些步骤。计算机比人类快得多，所以它们可以在更短的时间内执行这些步骤。计算的基本因素是速度。计算机无法做到与人类在本质上不同的事情，但它可以做得更快——快得多。算法在计算机上获得强大功能，因为它可以在计算机上执行相同步骤所需时间的一小部分内完成，但这些步骤仍然是相同的。

If an algorithm is a set of steps we can carry out ourselves, programming is the activity by which we write down the steps in the notation that the computer understands. Then it is the computer that will carry them out. Computers are much faster than human beings, so they can execute the steps in less time. The fundamental factor in computing is speed. A computer cannot do something qualitatively different from what we humans can do, but it can do it faster—a lot faster. An algorithm gains power on a computer because it can be executed there in a fraction of the time it would take us to perform the same steps, but they are still the same steps.

编程语言为我们提供了一种向计算机描述算法步骤的方法。它还提供了使用三种基本控制结构（序列、选择和迭代）来构建算法的方法。我们写出这些步骤并描述如何使用我们正在使用的特定编程语言提供的词汇和语法来编排它们。

A programming language gives us a way to describe to a computer the steps of algorithms. It also provides the means to structure them using the three fundamental control structures: sequence, selection, and iteration. We write the steps and describe how they are choreographed using the vocabulary and syntax provided by the particular programming language we are using.

除了速度快之外，使用计算机还有一个额外的优势：如果你还记得自己是如何学习执行长乘法和除法的，那可能要花很多时间练习，而且可能并不那么令人兴奋。正如我们上面提到的，这些东西在我们很小的时候就被灌输到脑子里，而灌输到脑子里可不是什么愉快的过程。计算机不会感到无聊，所以让它们执行算法的另一个原因是，它能消除我们的无聊，让我们有时间去做更有趣的事情。

There is an additional advantage to using computers apart from speed; if you can recall how you learned to perform long multiplication and division, it may have taken a lot of practice, and may not have been that exciting. As we noted above, these things are drilled into our heads at an early age, and drilling inside a head is not a pleasant procedure. Computers do not suffer from boredom, so an added reason to have them perform algorithms is to take out the tedium and leave us time to do more interesting things.

虽然算法通常在计算机上执行，但在用编程语言编写之后，它主要是为了人类而写的，人类必须理解算法的工作原理以及何时可以使用它。这就引出了即使是经验丰富的计算机科学家和经验丰富的程序员也会忘记的一个关键点。真正理解算法的唯一方法是手动执行。我们必须能够执行算法，就像计算机执行实现算法的程序一样。如今，我们有幸拥有种类繁多的媒体来帮助我们学习：只需点击一下即可观看精彩的视频、动画和模拟。所有这些都很棒，但当你遇到困难时，请准备好你的笔和便笺簿。这句话同样适用于这些线条。你真的理解如何创造节奏吗？你试过创建一个吗？你能找到 252 和 24 的最大公约数吗？

Although an algorithm is usually executed on a computer, after being written in a programming language, it is primarily written for humans, who must understand how it works and when it can be used. This brings us to something essential that even experienced computer scientists and seasoned programmers forget. The only way to truly understand an algorithm is to perform it by hand. We must be able to execute the algorithm, in the same way the computer would execute a program that implements it. At this date and time, we are privileged to have at our disposal an amazing array of media that can help us learn: superb videos, animations, and simulations are one click away. All these are great, but when you are stuck, have your pen and pad nearby. The same applies to these very lines. Have you really understood how you can create rhythms? Did you try to create one? Can you find the greatest common divisor of 252 and 24?

所有程序都会实现一系列步骤来完成某件事，所以我们可能会说所有程序都是算法。然而，我们要求更严格一些，希望我们的步骤满足以下几个特征：4

All programs implement a set of steps to do something, so we could be tempted to say that all programs are algorithms. We are a bit stricter, however, and want our steps to meet certain characteristics:4

1. 1. 步骤必须在有限步后终止。算法不可能永远运行下去。（只要运行程序的计算机还能运行，它就可以永远运行下去。但这种程序并非算法的实现，而只是一个计算过程。）
2. 1. The steps must terminate after a finite number of steps. An algorithm cannot run forever. (A program may run forever, as long as the computer on which it runs remains operational. That program would not be an implementation of an algorithm; it would just be a computational process.)
3. 2. 步骤必须精确，以便我们能够毫无混乱地执行。
4. 2. The steps must be precise, so that we can execute them without confusion.
5. 3. 算法可能对某些输入进行操作；就欧几里得算法而言，它对两个整数进行操作。
6. 3. The algorithm may operate on some input; in the case of Euclid’s algorithm, it operates on two integers.
7. 4. 算法有一些输出；这就是算法的全部目的：产生一些结果。在欧几里得算法中，这就是最大公约数。
8. 4. The algorithm has some output; that is the whole purpose of the algorithm: to produce something as a result. In Euclid’s algorithm, that is the greatest common divisor.
9. 5. 算法必须有效。人类应该能够用纸笔在合理的时间内完成每个步骤。
10. 5. The algorithm must be effective. A human should be able to execute each step in a reasonable amount of time with pen and paper.

这些特性确保算法能够完成某些任务。算法之所以存在，是因为它能够完成一些有用的任务。一些看似无关紧要的算法确实存在，计算机科学家也可能出于玩笑或失误而发明一些毫无用处的算法，但我们真正感兴趣的是那些对我们有用的算法。在使用算法时，仅仅证明算法可以完成某些任务是不够的。我们希望算法具有实用价值，为此，它们必须能够出色地完成某些任务。

These characteristics ensure that the algorithm does something. An algorithm exists because it does something useful. Frivolous algorithms do exist, and computer scientists may invent useless algorithms either in jest or by mistake, but we are really interested in algorithms that have some utility to us. When working with algorithms, it is not enough to show that something can be done. We want algorithms to be of practical interest, and for that purpose they must do something well.

这就是算法和数学之间的根本区别。早期的计算机科学家大多是数学家，计算机科学也大量运用数学，但它并非一门数学学科。数学家想要证明某事成立，而计算机科学家则想要让它发挥作用。

Therein lies a fundamental difference between algorithms and mathematics. Most early computer scientists were mathematicians, and computer science uses a lot of mathematics, but it is not a mathematical discipline. A mathematician wants to prove that something is so; a computer scientist wants to make it work.

算法的首要特征是它应该需要有限数量的步骤。这并不十分精确。我们不希望只有有限数量的步骤。我们希望步骤数量足够少，以便在实践中执行，从而确保我们的算法能够在合理的时间内完成。这意味着仅仅提出一个算法是不够的；该算法还必须在实践中有效。让我们看一个例子来说明了解某件事和知道如何高效地做某件事之间的区别。假设我们有一个如下的网格：

Our first characteristic of an algorithm is that it should require a finite number of steps. That is not very precise. We do not want to have just a finite number of steps. We want to have a number of steps that is small enough to execute them in practice, so that our algorithm finishes in a reasonable amount of time. That means that coming up with an algorithm is not enough; the algorithm must also be effective in practice. Let’s see an example to illustrate the difference between knowing something and knowing how to do something efficiently. Imagine we have a grid like the following:

我们希望找到从网格左上角到右下角的最短路径，且路径中不会重复访问同一个点。每条路径的长度等于网格上点之间的连接数。这里有一种方法：找到所有这样的路径，测量每条路径的长度，然后选取最短的一条，如果长度相同，则取其中任意一条最短的路径。路径总数为 12 条，如下所示：

We want to find the shortest path from the upper-left corner of the grid to the lower-right corner, without visiting the same place twice. The length of each path is equal to the number of links between points on the grid. Here is one way to do it: find all such paths, measure how long each of them is, and pick up the shortest, or any of the shortest in case of ties. The total number of paths is 12, which you can see below:

有五条长度为 4 的路径，因此我们可以选择其中任何一条。

There are five paths of length 4, so we can pick any one of them.

不过，我们并不局限于网格。我们可以拥有、，甚至更大的网格。然后我们发现，我们的方法扩展性不佳。从网格的左上角到右下角有 184 条路径；如果我们扩展到网格本身，这样的路径数量就会增加到 8,512 条。路径的数量还在持续快速增长——事实上，增长速度越来越快——并且甚至数出这样的路径都是一件难事。当我们到达一个网格时，我们会得到 8,402,974,857,881,133,471,007,083,745,436,809,127,296,054,293,775,383,549,824,742,623,937,028,497,898,215,256,929,178,577,083,970,960,121,625,602,506,027,316,549,718,402,106,494,049,978,375,604,247,408 条路径。这个数字有 151 位十进制数字，是通过一个执行算法的计算机程序得出的；是的，我们使用一种算法来理解另一种算法的行为。5

We are not limited to grids, though. We can have , , and even larger grids. Then we discover that our method does not scale well. There are 184 paths from the upper-left corner to the bottom-right corner of a grid; if we go to the grid, the number of such paths increases to 8,512. The number of paths continues to increase apace—in fact, at ever larger paces—and even counting such paths is a challenge. When we reach a grid, we get 8 402 974 857 881 133 471 007 083 745 436 809 127 296 054 293 775 383 549 824 742 623 937 028 497 898 215 256 929 178 577 083 970 960 121 625 602 506 027 316 549 718 402 106 494 049 978 375 604 247 408 paths. This number has 151 decimal digits and was found with a computer program implementing an algorithm; yes, we use an algorithm to understand the behavior of another algorithm.5

枚举所有路径并挑选最短路径的过程无疑是正确的，并且总能找到最短路径——或者，如果存在多条同样短的路径，则可以找到任意一条最短路径——但这显然是不切实际的。而且，它完全没用，因为有些算法无需枚举所有可能的路径就能找到最短路径，从而节省大量时间，并使我们能够处理任何大小的网格。在网格中，找到答案所需的步骤只有数百个；我们将在下一章中看到。

The procedure for enumerating all paths and picking the shortest one is undoubtedly correct, and will always give us the shortest path—or any of the shortest paths, if there are many equally short ones—yet it is definitely impractical. Also, it is completely useless, as there are algorithms that will find the shortest path without having to enumerate all possible paths, thus saving a lot of time and allowing us to tackle grids of any size. In the grid, the number of steps required to find the answer is only in the order of the hundreds; we’ll see it in the next chapter.

什么是实用算法，以及在何种意义上一种算法比其他算法更实用，这些问题是算法应用的核心。在本书的其余部分，我们将看到，解决同一问题通常存在不同的算法，我们会选择最适合特定应用场景的算法。与所有工具一样，某些算法比其他算法更适合特定情况。与许多其他工具不同，不过，我们拥有明确的方法来评估算法的优点。

The question of what is a practical algorithm and in what sense an algorithm is more practical than others is at the heart of any application of them. We’ll see in the rest of the book that there often exist different algorithms for solving the same problem and we choose the algorithm that is most appropriate for the application at each particular setting. Like all tools, some algorithms are more suitable for particular cases than others. Unlike many other tools, though, we possess a well-defined way to evaluate the merits of algorithms.

测量算法

Measuring Algorithms

当我们研究一种算法来解决问题时，我们想知道它的性能如何。速度始终是一个重要因素。我们使用计算机算法来比人类更快地完成任务。

When we are investigating an algorithm to solve a problem, we want to know how it is going to perform. Speed is always an important factor. We use algorithms on computers to do things faster than a human would do.

随着计算机硬件的改进，我们通常不再满足于了解实现算法的程序在特定计算机上的运行情况。我们的计算机可能比测试该算法的计算机更快或更慢，而且几年后，在过时的机器上对算法的测试结果将只具有历史意义。我们需要一种独立于计算机硬件来衡量算法性能的方法。

As computer hardware improves, we are usually not content with knowing how a program implementing an algorithm runs on a particular computer. Our computer may be faster or slower than the one that the algorithm was measured on, and after some years, measurements of algorithms on outdated machines will have only historical interest. We need a way to measure how well an algorithm performs independent of computer hardware.

然而，我们试图解决的问题的规模应该在某种程度上反映在我们衡量算法性能的方式上。我们并不真正关心排序 10 个项目需要多长时间；毕竟，我们可以手动完成。我们关心的是排序一百万个或更多项目需要多长时间。我们希望衡量一个算法在处理复杂问题时的表现。

The size of the problem we are trying to solve, though, should be somehow reflected in how we measure the performance of an algorithm. We don’t really care how long it takes to sort 10 items; after all, we can do that by hand. We care how long it takes to sort a million items or more. We want a measure of how we expect an algorithm to perform in problems that are not trivial.

为此，我们需要一种方法来量化算法中问题的规模。感兴趣的维度各不相同在不同问题之间。如果我们想在计算机中对一些项目进行排序，相关的维度就是我们要排序的项目的数量（而不是项目的大小或组成）。如果我们想将两个数字相乘，相关的维度就是这两个数字的位数（这对人类来说也合情合理：长乘法之所以长，是因为它取决于每个数字的位数）。当我们研究一个问题以及解决该问题的候选算法时，我们总是会考虑问题的规模。

To do that, we need a way to quantify the size of problems fed to algorithms. The dimension of interest varies among different problems. If we want to sort a number of items in our computer, the relevant dimension is the number of items that we want to sort (and not, say, the size or composition of the items). If we want to multiply two numbers, the relevant dimension is the number of digits of the two numbers (that also makes sense for humans: long multiplication is long because it depends on how many digits each number has). When we study a problem and candidate algorithms for tackling it, we do it always with the size of the problem under consideration.

虽然具体问题有不同的规模评估方法，但最终，我们会用一个整数n来指定每个问题的大小。继续上面的例子，n要么是待排序项的个数，要么是待乘数的位数。这样，​​我们就能讨论处理规模为n的算法的性能了。

Although particular problems have different ways to assess their size, in the end, for each problem we specify its size with an integer number, which we call n. Picking up the examples above, n is the number of either the items to sort or digits of the numbers we want to multiply. Then we want to be able to talk about the performance of algorithms tackling problems of size n.

算法所需的时间与其计算复杂度相关。算法的计算复杂度是指运行算法所需的资源量。这里主要有两种资源：时间（运行所需的时间）和空间（即需要多少计算机内存）。

The time required by an algorithm is related to its computational complexity. The computational complexity of an algorithm is the amount of resources it requires to run. There are two main kinds of resources here: time, how long it takes, and space, how much storage it requires in terms of computer memory.

我们现在关注的是时间。由于不同的计算机性能特征不同，讨论某个算法在特定计算机上运行所需的时间或许能让我们预判一下预期结果。在其他计算机上运行时，我们想要更通用的描述。计算机的速度取决于执行基本操作所需的时间。为了避免这种特殊性，我们选择讨论运行算法所需的操作数量，而不是在特定计算机上运行这些操作的实际时间。

We are focusing on time right now. As there are computers with different performance characteristics, talking about the time taken by an algorithm to run on a particular computer may give us some indication of what to expect when it runs on other computers, but we would like something more general. The speed of a computer depends on the time it takes to execute basic operations. To get around such specificities, we instead choose to talk about the number of operations required to run an algorithm, not the actual time it takes on a specific computer to run these operations.

话虽如此，请注意，我们可能有些滥用术语，将“操作”和“时间”视为同义词。虽然严格来说，我们应该说某个算法需要“ x 次操作”，但我们也会说该算法的“时间x ”，以表明它在实际运行该算法的任何计算机上的运行时间等于执行x 次操作所需的时间。即使实际时间会因硬件而异，但当我们想要比较在同一台计算机上分别在“时间x ”和“时间y ”运行的两个算法时，无论这台计算机是哪台，这都无关紧要。

Now having said that, note that we’ll be abusing terminology a bit and treating “operations” and “time” as synonyms. Although we should be strictly saying that an algorithm requires “x operations,” we’ll also be saying that the algorithm is “time x,” to indicate that it runs in the time required to execute x operations on any computer that the algorithm is actually run. Even though the actual time will vary with different hardware, it does not matter when we want to compare two algorithms that run on “time x” and “time y” on the same computer, whatever computer that is.

现在我们回到算法所给定问题的规模。由于我们感兴趣的是非平凡问题，所以我们不会关心小规模问题时会发生什么。我们关心的是达到一定规模后会发生什么。我们不会确切地说明这个规模是多少，但我们始终假设它是相当大的。

Now we return to the size of the problem given to an algorithm. As we are interested in nontrivial problems, we won’t care about what happens with small problem sizes. We will be concerned with what happens once we reach a certain size. We won’t say exactly what this size is, but we will always assume that it is substantial.

有一个复杂性的定义，在实践中被证明是有用的。它也有一个符号和名称。我们称之为大O符号。在大 O 符号中，在点的位置，我们写一个表达式。这个符号表示该算法所需的时间最多是表达式的倍数。让我们看看这意味着什么：

There is a definition of complexity that has proved to be useful in practice. It also has a symbol and name. We write and call it the big O notation. Inside the big O, in the place of the dot, we write an expression. The notation means that the algorithm will take time that is at most a multiple of the expression. Let us see what that means:

• • 如果你想在一系列项目（有n 个项目）中查找某个项目，并且项目序列没有任何顺序，则复杂度为。也就是说，对于n 个项目，在其中找到特定项目所需的时间不会超过项目数量的倍数。
• • If you want to look for something in a sequence of items—there are n items—and the sequence of items is not ordered in any way, the complexity is . That is, for n items, the time required to find a particular one in them will not be more than a multiple of the number of items.
• • 如果要使用长乘法将两个n位数字相乘，则复杂度为。也就是说，乘法所需的时间不会超过数字大小的平方的倍数。
• • If you want to multiply two n digit numbers using long multiplication, the complexity is . That is, the time required for the multiplication will not be more than a multiple of the square of the size of the numbers.

如果我们有一个具有复杂度的算法，那么对于 10,000 的输入规模，我们预计它需要数万个步骤。如果该算法具有复杂度，对于类似规模的输入，我们预计它需要一亿个步骤。对于许多问题来说，这个规模并不算大。计算机通常对 10,000 个项目进行排序。但你会发现，算法复杂度所代表的步骤数量规模可能会非常大。

If we have an algorithm that has complexity, then for an input size of 10,000 we expect it to need a multiple of ten thousand steps. If the algorithm has complexity, for a similarly sized input, we expect it to need a hundred million steps. For many problems, this is not a large size. Computers routinely sort 10,000 items. But you see that the scale of the number of steps represented by the algorithm’s complexity can grow large.

以下示例或许能帮助你理解我们将遇到的一些数字的大小。1000亿，也就是1后面有11个零的数字。如果你拿1000亿个汉堡，把它们首尾相连，就能绕地球216圈，还能去月球，然后回来。

Here are some examples that may help you appreciate the size of some numbers that we will encounter. Take the number 100 billion, or ; this is one with 11 zeros behind it. If you take 100 billion hamburgers and lay them end to end, you can circle the earth 216 times, go to the moon, and come back.

十亿通常被称为千兆，至少在计算机中是这样。十亿（或千兆）之后是万亿（或太拉），即1万亿。如果以每秒一个数字的速度计算，则需要31000年才能达到一万亿。再增加1000，就达到了千万亿（或千万亿）的数量；根据生物学家E.O.威尔逊的说法，地球上蚂蚁的总数在1到10千万亿之间。换句话说，地球上有1到10千万亿只蚂蚁。

A billion of something is usually called giga something, at least in computers. Next after the billion, or giga, comes the trillion, or tera, which is 1,000 billion, . If you start counting one number per second, you will need 31,000 years to get to one trillion. Up by 1,000 again and we get to one quadrillion, , or peta; the total number of ants that live on the earth is between 1 and 10 quadrillion, according to biologist E. O. Wilson. In other words, we have between 1 and 10 petaants on our planet.

千万亿之后是五千万亿，或称艾克萨；一千万亿是，大约是 10 个大海滩的沙粒数量。例如，10 个科帕卡巴纳海滩有 1 艾克萨格伦沙粒。再往上，我们得到了，即一千万亿，或称泽塔。可观测宇宙中的恒星数量是 1 泽塔星。在yotta之后我们已经找不到前缀了，它代表，一千万亿。但数字总是可以越来越大。这个数字叫做googol——是的，你可能知道一家公司就是用故意拼写错误的名字来命名的。然后是 10 的 googol 次方，即 ，也就是1 googolplex。6

After quadrillion comes quintillion, or exa; a quintillion is and is about the number of grains of sand in 10 large beaches. For example, 10 Copacabana Beaches have one exagrain of sand. Up again, we arrive at , one sextillion, or zetta. The number of stars in the observable universe is one zettastars. We run out of prefixes after yotta, which stands for , one septillion. But numbers can always get larger. The number is called a googol—yes, you probably know a company that has named itself after a purposeful misspelling. And then there is 10 raised to the googol power, , which is one googolplex.6

这些类比将有助于我们理解本书其余部分将要讨论的具体算法的相对优点。虽然理论上我们可以任何类型的复杂算法，我们通常处理的算法都分为几类不同的类型。

These analogies will help us appreciate the relative merits of specific algorithms that we will examine in the rest of the book. Although in theory we could have algorithms of any kind of complexity, the algorithms we usually deal with fall into few different groups.

复杂性家族

Complexity Families

所有算法中最快的一类是那些无论输入如何，运行时间都不超过常数级的算法。我们用 表示这种复杂度；例如，一个检查数字最后一位是奇数还是偶数的算法，不会受到数字大小的影响，并且运行时间是常数级。 中的 1表示该算法运行步骤不超过 1 的倍数，即步骤数为常数级。

The fastest family of all algorithms comprises the algorithms that run in no more than constant time, no matter what their input. We denote this complexity with ; for example, an algorithm that checks if the last digit of a number is odd or even will not be affected by the size of the number and will run in constant time. The 1 in follows from the fact that means that the algorithm needs no more than a multiple of one steps to run—that is, a constant number of steps.

在了解下一个复杂性家族之前，我们需要简要了解一下事物增长或收缩的一种特殊方式。如果将某项加法多次，就相当于将其乘以它。如果将某项乘以多次，就相当于将其乘以幂或指数。我们刚刚看到了像（或更多）这样的指数可以有多大。或许不太明显的是，指数增长会以多快的速度导致令人眼花缭乱的增长——这种现象被称为指数增长。

Before we meet the next complexity family, we need to take a brief excursion into a particular way things can grow or shrink. If you add something many times, you multiply it. If you multiply something many times, you raise it to a power or exponentiate it. We just saw how big numbers with exponents like (or more) can get. What is perhaps not immediately obvious is how quickly exponentiation leads to dizzying escalation—a phenomenon called exponential growth.

关于国际象棋发明的一个可能是杜撰的故事很能说明问题。国际象棋发明国的统治者问它的发明者，他希望一份礼物（可惜，这些故事里都是“他”）。他回答说，他希望在棋盘的第一个方格上放一粒米，第二个方格上放两粒，第三个方格上放四粒，以此类推。国王以为自己轻松脱身，满足了他的愿望。可惜，事情很快就变糟了。这个数列以2的幂次方增长：第一个方格，第二个方格，第三个方格，因此最后一个方格中的米粒数量将是，这是一个无论如何都无法达到的数量（它等于9,223,372,036,854,775,808，约等于9千万亿）。

The probably apocryphal story about the invention of chess is illustrative. The ruler of the country where chess was invented asked its inventor what he would like for a gift (alas, it is a “he” in these stories). He replied that he would like one grain of rice on the first square of the chessboard, two on the second, four on the third, and so on. The king thought that he got off easily and granted the wish. Unfortunately, things quickly turned sour. The sequence grows in powers of two: in the first square, in the second square, in the third square, and thus in the last square the number of grains would be , a quantity unreachable by any means (it is equal to 9,223,372,036,854,775,808, or about 9 quintillion).

指数增长也能帮助我们理解为什么将一张纸折叠多次如此困难。每次折叠，折叠纸的层数都会翻倍。折叠10次后，纸张就会变得厚厚的。如果纸张厚度为0.1毫米，那么现在折叠起来的纸团厚度将超过10厘米。除了将其对折所需的巨大力量之外，从物理角度来看，这几乎是不可能的，因为要折叠某样东西，它的长度必须大于厚度。7

Exponential growth can also help us understand why it is so difficult to fold a piece of paper many times. Each time you fold it, you double the number of layers of the folded paper. After 10 folds, you have layers. If your sheet is 0.1 millimeters thick, you now have a folded wad that is over 10 centimeters thick. Apart from the sheer force you will need to fold that in two, it may not be physically possible at all to do it, because to fold something it must be longer than thick.7

指数级增长是计算机性能逐年提升的原因。根据摩尔定律，集成电路中晶体管的数量大约每两年翻一番。该定律以创立仙童半导体公司和英特尔公司的戈登·摩尔的名字命名。他于 1965 年提出了这一观察；事实证明，该定律具有先见之明，因此，从 1971 年处理器中大约 2,000 个晶体管（英特尔4004）到 2017 年的 190 多亿（32 核 AMD Epyc）。8

Exponential growth is the reason why computers have gotten more and more powerful over the years. According to Moore’s law, the number of transistors in an integrated circuit doubles about every two years. The law is named after Gordon Moore, who founded Fairchild Semiconductor and Intel. He made the observation in 1965; the law proved prescient, so that we have gone from about 2,000 transistors in a processor in 1971 (the Intel 4004) to more than 19 billion in 2017 (the 32-core AMD Epyc).8

了解了增长之后，让我们来探索它的反面。如果某个数是倍数，可以用除法来逆转运算，得到原始值。如果某个数是幂，那么该如何逆转运算呢？幂的逆运算就是对数。

Having seen growth, let us explore now its opposite. If you have a multiple of something, you use division to reverse the operation and get the original value. If you have the power of something, , how do you reverse the operation? The reverse of raising to a power is the logarithm.

对数有时被认为是通俗数学和入门数学之间的分界线；甚至连它的名字都带着一丝难以理解的意味。如果对数看起来有些模糊，你需要记住，对数的运算是将数乘以幂的逆运算。就像我们乘幂时要反复乘一样，对数的运算也是反复除以。

Logarithms are sometimes taken as the boundary between mathematics for all and mathematics for the initiated; even the name has an aura of incomprehension. If logarithms seem somewhat hazy, you need to keep in mind that the logarithm of a number is the reverse of raising the number to a power. Just as when we raise to a power, we multiply repeatedly, when we take a logarithm, we divide repeatedly.

对数是对“我应该将一个数取几次方才能得到我想要的值？”这个问题的答案。我们要取的数称为对数的底数。所以，如果问题是“我应该将 10 取几次方才能得到 1000？”，答案是 3，因为。当然，我们可能想要取不同的数，也就是说，使用不同的底数。对数的符号是​​，它对应于这个问题“我应该将a取几次方才能得到x？”。当 时，我们只需去掉下标，因为以 为底的对数很常见，所以我们不写 ，而直接写。

The logarithm is the answer to the question, “To which power should I raise a number to get the value I want?” The number we are raising is called the base of the logarithm. So if the question is, “To which power should I raise 10 to get 1000?,” the answer is 3 because . Of course, we may want to raise a different number—that is, use a different base. The notation for logarithms is and it corresponds to the question, “To which power should I raise a to get x?” When , we just drop the subscript, because logarithms base 10 are common, so instead of writing we simply write .

还有另外两个常见的底数。当底数是数学常数e时，我们写为。数学常数e，称为欧拉常数，约等于 2.71828。在自然科学中我们经常遇到，这就是它被称为自然对数的原因。另一个常见的底数是 2，我们不写 ，而写为。以 2 为底的对数在计算机科学和算法中很常见，但在这些领域之外可能并不常用，尽管我们已经见过它们。在折纸中，如果一叠纸有 1,024 层，则它已被折叠了多次。在国际象棋的例子中，米粒的数量来自我们执行的加倍次数，即。

There are also two other common bases. When the base is the mathematical constant e, we write . The mathematical constant e, called Euler’s number, is approximately equal to 2.71828. In the natural sciences we meet a lot, which is why it is called natural logarithm. The other common base is 2, and instead of writing we write . Base 2 logarithms are common in computer science and algorithms, but probably unused outside these fields, although we have already met them. In paper folding, if a wad of paper has 1,024 layers, it has been folded times. In the chess example, the number of grains of rice results from the number of doublings we perform, which are .

我们在算法中经常看到“复杂度”的原因是，每当我们通过将一个问题拆分成两个大小相等的小问题来解决时，它就会出现；这被称为“分而治之”，其原理就像将一张纸对折。在已排序的一组项目中搜索某个内容的最有效方法是“复杂度” 。这相当惊人；这意味着，要在十亿个有序的项目中查找某个内容，你只需要对你的条目进行探测。

The reason we see a lot in algorithms is that it appears whenever we solve a problem by splitting it in two equal smaller problems; this is called divide and conquer, and it works like folding a sheet in two. The most efficient way to search for something in a sorted group of items has complexity . That is pretty amazing; it entails that to find something among one billion ordered items, you need only probes into your items.

具有对数复杂度的算法是继常数时间算法之后的最佳选择。接下来是运行时间的算法，它们被称为线性时间算法，因为它们的时间与n成比例增长；这意味着它们的增长速度是n的倍数。我们看到，在无序集合所需的时间与集合中元素的数量成正比。对比有序集合，复杂度增加了多少；组织问题的数据会对问题的解决方式产生重大影响。一般来说，如果算法必须读取问题的所有输入，线性时间是我们可以预期的最佳行为，因为这将需要读取n 个输入的时间。

Algorithms that have logarithmic complexity are the next best thing after algorithms that run in constant time. Next come algorithms that run in , which are called linear time algorithms because their time grows proportionally with n; that means that they grow as multiples of n. We saw that searching for an item in an unordered set of items requires time proportional to the number of the items, . See how the complexity increased compared to when the items are ordered; organizing the data of our problem can have a big impact on how it can be solved. In general, linear time is the best behavior we can expect of an algorithm if it has to read through all the inputs of the problem, as this will require time for n inputs.

如果我们将线性和对数时间结合起来，我们就得到了对数线性时间算法，它们的时间增长了n乘以其对数，即。最好的排序算法——即将项目按顺序排列——的复杂度为。这可能看起来有点令人惊讶；毕竟，可以证明，如果你有n 个项目，并且想要将每个项目与所有其他项目进行比较，则需要时间，这大于。9另外，如果你有n 个项目需要排序，那么你肯定需要时间来检查所有项目。对它们进行排序需要将该数字乘以一个小于n本身的因子。我们将在本书的后面看到如何做到这一点。

If we combine linear and logarithmic times, we get loglinear time algorithms, where their time grows by n multiplied by its logarithm, . The best algorithms for sorting—that is, putting items in order—have complexity . That may look a bit surprising; after all, it can be shown that if you have n items and want to compare each item with all other items, it requires time , which is bigger than .9 Also, if you have n items that you want to sort, you definitely need time to examine all of them. Sorting them requires multiplying that number by a smaller factor than n itself. We’ll see how this can be done, later on in the book.

下一个计算复杂度族是n 的常数次幂，即 ；这被称为多项式复杂度。多项式时间算法非常高效，除非k很大，但这种情况很少发生。当我们尝试解决计算问题时，如果能找到一个多项式时间算法，我们通常会很高兴。

The next computational complexity family is n raised to a constant power, ; this is called polynomial complexity. Polynomial time algorithms are efficient, except if k is big, but this rarely happens. When we try to solve a computational problem, we are usually delighted if we can come up with a polynomial time algorithm.

形式的复杂度称为指数复杂度。注意它与指数为常数的多项式复杂度的区别；这里的指数是变化的。我们看到了指数增长是如何爆炸式增长的。宇宙将无法存在足够长的时间去看到针对非平凡输入的指数算法的答案。从理论的角度来看，这样的算法很有趣，因为它们表明可以找到解决方案。然后，我们可以寻找复杂度更低的更好算法，或者我们也许能够证明找不到更好的算法，在这种情况下，我们可以接受一些不太理想的结果——例如近似解。

A complexity of the form is called exponential complexity. Note the difference with the polynomial complexity where the exponent was constant; here it is the exponent that changes. We saw how exponential growth explodes. The universe will not survive long enough to see the answer of exponential algorithms for nontrivial inputs. Such algorithms are interesting from a theoretical point of view because they show that a solution can be found. We can then search for better algorithms with lower complexity, or we may be able to prove that no better algorithms can be found, in which case we can settle for something less than the ideal—for instance, approximate solutions.

有一种东西的增长速度甚至比幂运算还要快，那就是阶乘。如果你以前没遇到过阶乘，那么自然数n的阶乘——我们写为n !——就是所有直到该数的自然数的乘积：。即使你没遇到过 100!，你很可能在不知情的情况下遇到过 52!。这是一副牌的不同洗牌次数。以阶乘来衡量运行时间的算法具有阶乘复杂度。

There is something that grows even faster than exponentiation, and this is the factorial. If you have not encountered a factorial before, the factorial of a natural number n—which we write as n!—is simply the product of all the natural numbers up to and including that number: . Even if you have not encountered 100! you probably have encountered 52! even without knowing it. That is the number of different shuffles of a deck of cards. Algorithms whose running time is measured in factorials have factorial complexity.

虽然像 100! 这样的数字可能看起来很奇特，但它在许多并不奇特的场景中都会出现，而不仅仅是纸牌游戏。例如，以下问题：“如果我们有一个城市列表，以及每对城市之间的距离，什么是访问每个城市一次并返回原城市的最短可能路线？"这被称为旅行商问题，解决这个问题的明显方法是检查所有城市的每条可能路径。不幸的是，对于n 个城市，这个数字是n！在 20 个城市之后，问题就变得难以处理了。有些算法比 稍好一些，但还不足以实用。对于这样一个简单的问题，这似乎令人惊讶，但我们能在可接受的时间内解决它的唯一方法是找到一个可能不是最优解，但足够接近最优解。许多其他具有重要实际意义的问题都是难以解决的——也就是说，我们不知道一个实用的算法来精确解决它们。即便如此，寻求越来越好的近似算法仍然是计算机科学中一个充满活力的领域。

Although numbers like 100! may seem exotic, they arise in many nonexotic settings and not just card games. Take, for example, the following problem: “If we have a list of cities and the distances between each pair of them, what is the shortest possible route that one should take to visit each city once and return to the origin city?” This is called the traveling salesman problem, and the obvious way to solve it is to examine every possible path taking in all cities. Unfortunately, for n cities this is n! The problem is unmanageable after, say, 20 cities. There are some algorithms that do it a bit better than , but not enough to be practical. Surprising as it may seem for such a straightforward problem, the only way we can solve it in an acceptable time is by finding a solution that may not be the optimal one, but is close enough to it. Many other problems of great practical importance are intractable—that is, we don’t know a practical algorithm to solve them exactly. Even so, the quest for better and better approximation algorithms is a vibrant field in computer science.

在下表中，您可以看到我们列出的复杂度族中，不同n值下各种函数的值。第一行给出了n值，也代表线性复杂度；后续行显示了复杂度递增的族。随着n 的增加，函数值也会增加，但增加的方式有所不同。函数n^ ^{3 的}值可以从一百万增加到一千万亿，但这与n^2或 100 相比根本不算什么！我们在行后留了一块空白，将实用算法与不实用算法区分开来。两者之间的界限是多项式算法，它正如我们所见，这些算法具有实际用途。复杂度较高的算法通常没有实际用途。

In the table that follows, you can see the value of various functions, falling under the complexity families we presented, for different values of n. The first row gives the n values and also stands in for linear complexity; subsequent rows show families of increasing complexity. As n increases, the function values increase, but the way they increase is different. The function n³ will take us from one million to one quintillion, but that is nothing compared to or 100! We have left a blank like after the row, separating practical from impractical algorithms. The border between the two are the polynomial algorithms, which as we saw are of practical use. Algorithms with higher complexity are usually not of practical use.

从图到算法

From Graphs to Algorithms

图的定义很广泛，它可以涵盖所有可以表示为相互连接的事物。图可能与某个地方的拓扑结构有一定关联，但节点和链接可能与空间中的点无关。

The definition of a graph is wide so that it can encompass everything that can be represented as things connected to other things. The graph may have some relevance to the topology of a place, but the nodes and links may have nothing to do with points in space.

社交网络就是这种图谱的一个例子。在社交网络中，节点代表社交参与者（可能是个人或组织），链接代表他们之间的互动。社交参与者可能是现实世界中的演员，链接可能是他们在电影中的合作。社交参与者也可能是我们自己，链接可能是我们在社交网络应用中与其他人之间的联系。我们可以利用社交网络来寻找人们的社群，前提是社群是由相互互动的人们组成的。目前已有算法能够在包含数百万个节点的图中高效地找到社群。

A social network is an example of such a graph. In a social network, nodes are social actors (these may be individuals or organizations), and the links represent interactions between them. The social actors may be real-world actors, and the links may be their collaborations in films. The social actors can be us, and the links may be our connections to other people in a social network application. We can then use social networks to find communities of people, starting from the premise that communities are formed by people who interact with each other. There exist algorithms that are able to find efficiently communities in graphs with millions of nodes.

柯尼斯堡图中的边是无向的，这意味着我们可以双向遍历它们；例如，我们可以从 A 到 B，也可以从 B 到 A。社交网络中，当连接是互惠的时，这种情况就出现了。但这并非总是必要的。根据我们的应用，图中的边可能是有向的。当这种情况发生时，我们会在边的两端画上箭头。有向图简称为有向图。您可以在下面看到一个有向图。请注意，这不是多重图；从 A 到 B 的边与从 B 到 A 的边不同。

The edges in the Königsberg graph are not directed, meaning that we can traverse them both ways; for example, we can go from A to B and B to A. The same goes for social networks, when the connections are reciprocal. That is not always necessary. Depending on our applications, edges in a graph may be directed. When this happens, we draw the edges with arrows at their ends. Directed graphs are called digraphs for short. You can see a digraph below. Note that this is not a multigraph; the edge from A to B is not the same as the edge from B to A.

万维网就是一个（巨大的）有向图的例子。我们可以用节点代表网页，用边代表每对网页之间的超链接来表示万维网。这个图是有向图，因为一个页面可能链接到另一个页面，但另一个页面不一定链接回第一个页面。

The World Wide Web is an example of a (huge) directed graph. We can represent the web with nodes standing in for web pages and edges standing in for the hyperlinks between each pair of pages. This graph is a directed graph, because a page may link to another page, but that other page does not necessarily link back to the first page.

如果可以从图中的某个节点出发，遍历边，然后回到起始节点，我们就说该图存在环。并非所有图都有环。柯尼斯堡图就有环——尽管它确实没有欧拉回路。科学史上一个著名的循环图（实际上是多重图）是奥古斯特·凯库勒的苯分子结构模型：5

When it is possible to start from a node in a graph, traverse edges, and come back to the node we started from, we say that the graph has a cycle. Not all graphs have cycles. The Königsberg graph has cycles—although it does not have a Eulerian circuit. A famous cyclic graph (actually a multigraph) in the history of science is August Kekulé’s model of the molecular structure of benzene:5

无环的图称为无环图。有向无环图是一类重要的图，我们通常称之为有向无环图 (DAG)。有向无环图有很多用途；例如，我们用它们来表示任务之间的优先级（任务是节点，优先级是它们之间的链接）、依赖关系、先决条件以及其他类似的安排。现在我们先不讨论无环图，转而关注有环图，它将为我们开启图算法的大门。

A graph without a cycle is called an acyclic graph. Directed acyclic graphs form an important class of graphs. We usually call them dags. Dags have many uses; for example, we use them to represent priorities between tasks (tasks are nodes, and priorities are links between them), dependency relations, prerequisites, and other similar arrangements. We’ll leave aside acyclic graphs now and turn our attention to cyclic graphs, which will provide us with a first window on algorithms on graphs.

路径和 DNA

Paths and DNA

过去几十年来最重要的科学发展之一是人类基因组的破译。由于在这一努力中开发的技术，我们现在可以研究遗传疾病、检测突变、研究灭绝物种的基因组以及其他令人着迷的应用。

One of the most important scientific developments of the last decades has been the decoding of the human genome. Thanks to the techniques that were developed in that effort, we can now investigate genetic diseases, detect mutations, and study genomes of extinct species, among other fascinating applications.

基因组编码于DNA中，DNA是一种由双螺旋结构构成的大型有机分子。双螺旋结构由四种碱基组成：胞嘧啶 (C)、鸟嘌呤 (G)、腺嘌呤 (A) 和胸腺嘧啶 (T)。双螺旋结构的每一部分都由一系列碱基构成，例如ACCGTATAG。双螺旋结构的另一部分由碱基构成，这些碱基根据AT和CG规则与第一部分上的相应碱基连接。因此，如果螺旋结构的一部分是ACCGTATAG，那么另一部分将是TGGCATATC。

Genomes are encoded in the DNA, a large organic molecule that is composed of a double helix. The double helix is made up of four bases: cytosine (C), guanine (G), adenine (A), and thymine (T). Each part of the double helix is constructed from a series of bases, like ACCGTATAG. The other part of the double helix is constructed from bases that are connected with their corresponding bases on the first part, according to the rules A-T and C-G. So if one part of the helix is ACCGTATAG, the other part will be TGGCATATC.

为了确定未知DNA片段的组成，我们可以按照以下方法进行。我们复制多条DNA链，并将它们分解成小片段——例如，每个片段包含三个碱基。使用专门的仪器，我们可以轻松识别这些小片段。这样，我们最终会得到一组已知片段。接下来，我们面临的问题是将这些片段组装成一个DNA序列，从而确定其组成。

In order to find the composition of an unknown DNA piece, we can work as follows. We create many copies of the chain and break them up into little fragments—for instance, fragments containing three bases each. Using specialized instruments, we can identify such small fragments easily. In this way we end up with a set of known fragments. We are then left with the problem of assembling the fragments into a DNA sequence, whose composition we will then know.

假设我们有以下片段，或者说是已知的聚合物：GTG、TGG、ATG、GGC、GCG、CGT、GCA、TGC、CAA 和 AAT。每个片段的长度都是 3；为了找到 DNA 序列，将它们分解后，我们创建一个图。在该图中，顶点是由长度为三的聚合物衍生而来的长度为二的聚合物，对于每种长度为三的聚合物，取前两个和后两个聚合物。因此，从 GTG 我们可以得到 GT 和 TG，从 TGG 我们可以得到 TG 和 GG。在该图中，我们为用于衍生这两个顶点的每种初始聚合物（长度为三的聚合物）添加一条边。我们将聚合物的名称赋予该边。从 ATG 我们可以得到顶点 AT 和 TG 以及边 ATG。您可以看到由我们的示例生成的图：

Suppose then that we have the following fragments, or polymers as they are known: GTG, TGG, ATG, GGC, GCG, CGT, GCA, TGC, CAA, and AAT. Each one of them has a length of three; to find the DNA sequence from which they were broken up, we create a graph. In that graph, the vertices are polymers of length two that are derived from the polymers of length three, taking for each polymer of length three the first two and last two polymers. So from GTG we will get GT and TG, and from TGG we will get TG and GG. In the graph, we add one edge for every one of the initial polymers or length three that was used to derive the two vertices. We give the name of the polymer to that edge. From ATG we get vertices AT and TG and the edge ATG. You can see the graph that results from our example:

利用我们创建的图，我们只需要找到一条恰好访问所有边一次的路径——即欧拉回路——就能找到初始的 DNA 序列。用于在图上寻找欧拉回路的 Hierholzer 算法由德国数学家 Carl Hierholzer 于 1873 年发表，其代码如下：6

With the graph we have created, we only need to find a tour in the graph that visits all edges exactly once—that is, an Eulerian circuit—in order to find the initial DNA sequence. The Hierholzer algorithm for finding Eulerian circuits on graphs was published by the German mathematician Carl Hierholzer in 1873 and goes like this:6

1. 1. 我们选择一个起始节点。
2. 1. We pick a starting node.
3. 2. 我们从一个节点走到另一个节点，直到回到起始节点。我们追踪到此点的路径不一定覆盖所有边。
4. 2. We go from node to node until we return to the starting node. The tour that we have traced to this point does not necessarily cover all edges.
5. 3. 只要存在一个顶点，它既属于我们已追踪的路径，又属于一条不在路径上的边，我们就从该顶点开始另一条路径，使用尚未使用的边，直到回到该顶点，形成另一条路径。然后，我们将这条路径与已追踪的路径拼接起来。
6. 3. As long as there exists a vertex that belongs to the tour we have traced, but is also part of an edge that is not in the path, we start another path from that vertex, using edges that we have not used yet, until we return to it, forming another tour. Then we splice this tour to the tour we have already traced.

如果我们使用示例图中的算法，我们将找到下图中的路径：

If we use the algorithm in our example graph, we will find the path in the following figure:

我们从 AT 出发，完成了 AT TG GG GC CA AA AT的路径。我们完成了一次路径，但并未覆盖所有边。我们发现 TG 有一条边 TGC，我们尚未覆盖。因此，我们前往 TG，并沿着 TGC 边开始路径，得到 TG GC CG GT TG。我们将第二条路径与第一条路径拼接，得到图中所示的路径：AT TG ( GC CG GT TG) GG GC CA AA AT。如果我们沿着生成的路径从第一个节点走到最后一个节点，无需踩到最后一个顶点，并连接所有顶点，保留它们仅有一次共同碱基，我们得到了DNA序列ATGCGTGGCA。你可以验证这个序列是否包含了我们一开始的所有聚合物；如果在到达序列末尾后绕回并回到开头，就会找到CAA和CAT。

We started from AT and made the tour AT TG GG GC CA AA AT. We made a tour, but we did not cover all the edges. We see that TG has an edge, TGC, that we have not covered yet. So we go to TG and do a tour starting along the TGC edge, getting TG GC CG GT TG. We splice the second path into the first, getting the one in the figure, AT TG ( GC CG GT TG) GG GC CA AA AT. If we walk the resulting path from the first node to the last, without stepping on the last, and concatenate the vertices keeping their common base only once, we get the DNA sequence ATGCGTGGCA. You can verify that this sequence contains all the polymers with which we started; CAA and CAT are found if you wrap around when you reach the end of the sequence and go to the beginning.

在这个特定的例子中，我们只找到了一条额外的路径，并将其拼接到原始路径中。通常情况下，可能会有更多路径；只要还有我们尚未覆盖的顶点及其边，算法就会重复第 3 步。Hierholzer 算法速度很快：如果实现得当，它可以在线性时间内运行，其中n是图中的边数。7

In this particular illustration, we only found one additional tour that we spliced into the original one. In general, there may be more; step 3 of the algorithm is repeated as long as there are vertices with edges that we have not covered yet. Hierholzer’s algorithm is fast: if implemented properly, it runs in linear time, , where n is the number of edges in the graph.7

安排锦标赛

Scheduling a Tournament

假设你正在组织一场锦标赛，参赛者将以双人赛的形式进行比赛，因此我们将进行一系列比赛。我们有八名参赛者，每位参赛者将参加四场比赛。我们的问题是如何安排锦标赛。我们希望安排比赛，使每位参赛者每天只参加一场比赛。

Suppose you are organizing a tournament in which the contestants will compete in pairs, so we’ll have a series of matches. We have eight contestants, and each contestant will play four matches. Our problem is how to schedule the tournament. We want to schedule the matches so that each contestant plays only one match per day.

一个显而易见的解决方案是每天只进行一场比赛，并允许锦标赛持续到需要的时间。由于我们有八名选手，每位选手参加四场比赛，因此锦标赛将持续16天。（；我们除以二，以免每场比赛重复计算）。我们将八位参赛者命名为 Alice、Bob、Carol、Dave、Eve、Frank、Grace 和 Heidi。这样我们只需使用他们名字的首字母即可识别他们。

An obvious solution is to have just one match per day and allow the tournament to last as long as needed. As we have eight contestants and each contestant plays four matches, the tournament would roll out over 16 days (; we divide by two so as not to count each match twice). We’ll name the eight contestants Alice, Bob, Carol, Dave, Eve, Frank, Grace, and Heidi. This allows us to use only the initial letter of their names to identify them.

如果我们将这个问题建模成图，就能找到更好的解决方案。我们为每位球员设置一个顶点，为每场比赛设置一条边。这样，图看起来就会像下面左侧那样。在右侧，我们用相应比赛的日期标记了边。我们是如何找到这个解决方案的？

We can find a better solution if we model the problem as a graph. We’ll have a vertex for each player and an edge for each match. Then the graph will look like the one on the left below. On the right, we have labeled the edges with the day on which the corresponding match will take place. How did we find this solution?

我们同意将比赛日期按顺序编号。比赛从第零天开始。我们将逐一安排所有比赛。

We agree to number the tournament days consecutively. Let the tournament start on day zero. We’ll schedule all matches, one by one.

1. 1. 选择一场我们尚未安排的比赛。如果所有比赛都已安排完毕，则停止。
2. 1. Take a match that we have not scheduled yet. If we have scheduled all matches, stop.
3. 2. 将比赛安排在最早的一天，这样两名选手当天都不会有其他比赛。返回步骤 1。
4. 2. Schedule the match on the earliest day so that neither of the two players has another match on that day. Return to step 1.

这个算法看似简单，你可能会怀疑它是否真的能解决我们的问题。那么，让我们逐步讲解一下，看看会发生什么。在下表中，我们可以逐一查看比赛情况，以及每场比赛的安排日期，并将算法应用到图表上。你应该先阅读表格的前两列，然后再阅读后两列：

This algorithm looks deceptively simple, and you may doubt that it really solves our problem. So let’s walk through it and see what happens. In the following table we can see the matches, one by one, and the day on which we schedule each match, as we apply the algorithm on the graph. You should read the first two columns of the table and then the next two:

我们首先进行 Alice 对阵 Bob 的比赛。Alice 和 Bob 在第零天（也就是我们分配比赛的那一天）都没有参加任何其他比赛。

We start by taking the match Alice versus Bob. Neither Alice nor Bob play any other match on day zero—that is, the day on which we’ll assign the match.

然后，我们选取​​另一场尚未安排的比赛——比如 Alice 对阵 Dave。虽然没有强制要求，但我们会按照字典顺序选取比赛选手。不过，我们也可以用其他任何方式，甚至是随机的，只要每场比赛只处理一次即可。Alice 已经安排了第 0 天的比赛，所以最早可以参加比赛的日期是第一天。

We then take another match we have not scheduled yet—say, Alice versus Dave. Although there is no requirement to do so, we’ll take the match players in lexicographical order as we continue, but bear in mind that we could take them in any other way, even randomly, as long as we treat each match only once. Alice already has a match scheduled on day zero, so the earliest available day for the match is day one.

接下来是爱丽丝和伊芙的比赛。爱丽丝在第0天和第一天都订了名，所以我们会安排在第二天。爱丽丝的最后一场比赛是和海蒂的；爱丽丝在第0天、第一天和第二天都订婚了，所以这场比赛只能安排在第三天了。

Next comes the match between Alice and Eve. Alice is booked on day zero and day one, so we’ll schedule it on day two. Alice’s final match will be with Heidi; Alice is engaged on days zero, one, and two, so this will have to go on day three.

我们已经安排好了 Alice 的比赛。接下来是 Bob 的比赛，除了我们已经安排好的与 Alice 的比赛之外，我们需要安排 Bob 对阵 Carol。Bob 已经安排在第零天（与 Alice 一起），所以这场比赛必须在第一天进行。安排 Bob 对阵 Eve 时，我们注意到 Bob 已经在第零天和第一天有事（我们刚刚安排好），而 Eve 计划在第二天与 Alice 比赛；因此，我们安排 Bob 对阵 Eve 的比赛在第三天。说到 Bob 对阵 Frank，Bob 在第零天和第一天都有比赛，但第二天有空，而 Frank 到目前为止还没有比赛。所以 Bob 对阵 Frank 的比赛在第二天进行，比 Bob 对阵 Eve 的比赛早。

We are done with Alice. Moving on to Bob’s matches, except for the one with Alice, which we have already scheduled, we need to plan Bob versus Carol. Bob is already scheduled on day zero (with Alice), so this match will have to go on day one. Scheduling Bob versus Eve, we notice that Bob is already engaged on day zero and day one (we just scheduled that), while Eve is scheduled to play on day two with Alice; we therefore schedule Bob versus Eve on day three. Going to Bob versus Frank, Bob has matches on days zero and one, but is free on day two, while Frank has no matches at all as of yet. So Bob versus Frank goes on day two, earlier than Bob versus Eve.

Bob 结束后，我们将安排 Carol 的比赛。Carol 和 Dave 在第 0 天都没有安排比赛，所以 Carol 对阵 Dave 的比赛将在锦标赛第一天进行。之后，Carol 对阵 Frank 的比赛可以安排在第三天进行，因为 Carol 会在第 0 天（我们刚刚安排好）和第一天（与 Bob 比赛，之前已经安排好）进行比赛，而 Frank 会在第二天与 Bob 比赛（之前也安排好）。Carol 对阵 Grace 的比赛将在第二天早些时候进行，因为 Grace 目前没有其他比赛安排，而 Carol 在第二天仍然有空。

After Bob, we’ll deal with Carol’s matches. Neither Carol nor Dave have a match scheduled on day zero, so Carol versus Dave will go on the first day of the tournament. After this, the Carol versus Frank match can take place on day three, because Carol plays matches on day zero (we just arranged that) and day one (with Bob, arranged previously), while Frank plays with Bob on day two (also arranged previously). Carol versus Grace will take place earlier, on day two, as Grace has no other matches planned as of yet and Carol is still free on day two.

我们对其余比赛也采取了类似的方法；有趣的是，图表内外方格内的比赛最早会在前两天进行。这是两个不同的小组在比赛开始之前同时进行的。最终，我们找到的解决方案比需要16天的简单解决方案有了显著的改进；我们只需要4天！

We proceed similarly with the rest of the matches; it is interesting that the matches in the inner and outer squares of the graph will happen as early as the first two days. These are two different groups playing in parallel before they start playing between them. At the end, the solution we find is a significant improvement over the naive solution requiring 16 days; we only need four!

这个锦标赛赛程安排问题实际上是一个更普遍问题的实例：边着色问题。图的边着色是指为边分配颜色，使得任何两条相邻边的颜色都不相同。这里应该用象征性的方式理解颜色。在我们的例子中，颜色是天数；通常，它们可以是任何其他不同的值集合。如果我们想要着色的不是边，而是图的顶点，使得由一条边连接的任何两个顶点的颜色都不相同，那么我们就有了顶点着色问题。边和顶点着色属于更广泛的图着色问题，这并不奇怪。

This tournament scheduling problem is in fact an instance of a more general problem: the edge coloring problem. An edge coloring of the graph is an assignment of colors to edges so that no two adjacent edges have the same color. Now color should be taken figuratively here. In our example, the colors are the days; in general, they can be any other set of distinct values. If instead of the edges, we want to color the vertices of the graph so that no two vertices that are linked by an edge share the same color, then we have the vertex coloring problem. Both edge and vertex coloring belong to the wider class of, no surprise, graph coloring problems.

我们描述的边着色算法简单高效（它逐一处理每条边，并且只处理一次）。这就是所谓的贪婪算法。贪婪算法试图通过在每个阶段寻找最佳解（而不是总体最优解）来解决问题。贪婪算法在很多问题中都很有用，因为在解决方案的每个阶段我们都需要做出选择，而我们的规则是“现在看起来最好的”。这种在算法演进过程中指导我们选择的策略称为启发式算法，源自希腊语heuriskein，意思是“寻找”（也就是找到一个解）。

The algorithm we described for edge coloring is simple and efficient (it takes each edge one by one, and only once). It is a so-called greedy algorithm. Greedy algorithms are algorithms that try to solve a problem by finding the best solution at each stage, not the optimal solution in general. Greedy algorithms are useful in many problems when at each stage of the solution we have a choice to make and our rule is “what looks best now.” Such strategies that guide our choices in the evolution of an algorithm are called heuristics, from the Greek heuriskein, which means “to find” (a solution, that is).

稍加思考，我们就能意识到，在算法中，就像在现实生活中一样，当下看起来最好的策略未必就是最好的。延迟满足或许会有回报；当下最好的选择，或许会让我们陷入日后后悔的陷阱。想象一下你正在爬山。贪婪的启发式算法会在每个点选择最陡峭的路径（我们假设你的攀爬能力无与伦比）。这并不一定会带你到达顶峰：它很可能会把你带到一个高原，从那里出发，唯一的出路就是返回。真正的登顶之路，或许是穿过一些较为平缓的山坡。

With some thought we can realize that in algorithms, as in real life, what looks best right now may not really be the best strategy. It may pay off to delay gratification; the best choice right now may lead us to a trap that we’ll regret later on. Imagine you are climbing a mountain. The greedy heuristic would be to select the steepest path at each point (we assume that your climbing prowess is unparalleled). This will not necessary lead you to the top: it may well lead you to a plateau, from which the only way is back. The real way to the top may lie through gentler slopes.

攀爬的比喻在计算机科学中经常用于解决问题。我们将问题建模，使得解决方案位于所有可能步骤的“顶端”。能够做出并尝试找到正确的动作；这被称为爬山法。当我们到达某个类似高原的地方时，我们说我们达到了局部最优，而不是全局最优，也就是我们追求的最高峰。

The climbing metaphor is frequently used in problem solving in computer science. We model our problem so that the solution lies at “the top” of the possible moves we can make and try to find the correct moves; this is called a hill climbing approach. When we arrive at something like a plateau, we say we arrived at a local optimum, but not the global optimum, the highest peak that we are after.

从爬山法回到锦标赛安排，我们为每场比赛选择了第一个可用的日期。不幸的是，这可能不是安排所有比赛的最佳方式。事实上，事实证明，图着色是一个难题。我们给出的算法不能保证给出最优解，即需要最少天数（或一般来说，颜色）的解决方案。与节点相邻的边数称为其度。可以证明，如果图中任何节点的最大度为d，则最多可以用d或种颜色为边着色；图的边所需的颜色数称为其色度。在我们的特定示例中，解决方案是最优的，并且我们使用了四天。然而，我们的算法可能无法在其他图中找到最优解。它可能会给我们一个比这更糟糕的解决方案。贪婪图着色的好处是我们知道该解决方案可能有多远：它给出的解决方案可能需要最多颜色，而不是d，但不会比这更糟糕。

From hill climbing back to tournament scheduling, we selected the first available day for each match. Unfortunately, this might not be the best way to schedule all matches. Indeed, it turns out that graph coloring is a difficult problem. The algorithm that we gave is not guaranteed to give the optimal solution—that is, the solution requiring the smallest number of days (or colors, in general). The number of edges adjacent to a node is called its degree. It can be proven that if the largest degree of any node in the graph is d, the edges can be colored with at most d or colors; the required number of colors for the edges of a graph is called its chromatic index. In our particular example, the solution is optimal, , and we used four days. Our algorithm, however, may not be able to find the optimal solution in some other graph. It may give us a solution worse than that. The good thing about greedy graph coloring is that we know how far off that solution might be: the solution it will give may need up to colors, instead of d, but no worse than that.

如果您想了解这是如何发生的，请考虑一个由连接到中心节点的“星星”组成的图表，如下页所示：

If you want to see how this may happen, consider a graph that consists of “stars” connected to a central node, like the one on the next page:

如果我们有k 个星星，每个星星有k条边加上一条到中心节点的边，我们首先给星星上色，我们会用k种颜色给星星的边上色。然后我们还需要k 种额外的颜色将星星连接到中心节点。总共需要 2k种颜色。这就是我们在左边所做的。但这不是最优解。如果我们首先给连接星星和中心节点的边上色，那么我们需要k 种颜色。然后我们可以只使用一种额外的颜色来给星星本身上色，总共需要 种颜色。你可以在右边看到我们如何做到这一点。所有这些都符合理论，因为每个星星的度数为。

If we have k stars, where each star has k edges plus an edge to the central node, and we start by coloring the stars, we’ll use k colors to color the edges of the stars. Then we’ll need k additional colors to connect the stars to the central node. The total is 2k colors. This is what we did on the left. But this is not the optimal solution. If we start by coloring the edges connecting the stars to the central node, we’ll need k colors for that. Then we can color the stars themselves using only one additional color, for a total of colors. You can see how we can do that on the right. All this is in accordance with theory, as each star has degree .

问题在于，贪婪算法决定以一种最终并非最优的方式（或者用正确的术语来说，以一种并非全局最优的方式）对边进行着色。它可能会找到最佳解决方案，也可能不会。不过，与最优解的差距也不是很大。这让人松了一口气，因为图的着色非常困难，如果我们想要一个能够为每个图找到最佳解的精确算法，该算法的复杂度将呈指数级增长，约为，其中n是图中的边数。因此，精确的边着色算法是无用的，除非图非常小。

The problem is that the greedy algorithm decides to order the edges to color in a way that is not optimal at the end, or to use the proper terminology, in a way that is not globally optimal. It might hit on the best solution, but it might not. Then again, the difference from the optimum solution is not that great. That is a relief because graph coloring is so difficult that if we want an exact algorithm that can find the best solution for every graph, the algorithm will have exponential complexity, about , where n is the number of edges in the graph. Exact edge coloring algorithms are therefore useless, except for tiny graphs.

我们提出的贪婪算法除了实用之外，还有一个额外的优点。它是一种在线算法：即使输入在开始时未知，而是随着算法的进展而出现，该算法也能正常工作。我们无需知道所有边即可开始运行该算法。即使在运行算法时，图是以逐个边的方式构建的，算法也能正常工作。如果在我们开始安排比赛之后，仍有球员报名参加比赛，就会出现这种情况。我们可以随时为每条边（比赛）着色，并且无论图何时完成，我们都会准备好边着色。此外，如果图以这种方式逐步创建，则此贪婪算法是最优算法；如果图是在解决问题的同时构建的，则根本不存在精确的算法，无论多么低效。8

The greedy algorithm we have presented has one additional nice property (apart from being practical). It is an online algorithm: an algorithm that works even if the inputs are not known when we start but instead arrive on the scene as we go. We don’t need to know all the edges to start running the algorithm. The algorithm will work correctly, even if the graph is constructed in a piecemeal fashion, one edge at a time, while we are running the algorithm. This would happen if players are signing up for the tournament even after we have started scheduling the matches. We will be able to color each edge (match) as it comes, and whenever the graph is finished, we’ll have an edge coloring ready. Moreover, this greedy algorithm is the optimum algorithm if the graph is created incrementally in this way; no exact algorithm, no matter how inefficient, exists at all when the graph is constructed while we are solving the problem.8

最短路径

Shortest Paths

正如我们所见，贪婪算法的工作原理是每一步都做出最佳决策——而这决策可能并非整体最佳。它带有某种机会主义的性质，或者说是“及时行乐”的感觉。不幸的是，正如伊索寓言所言，一只只活在当下的蚱蜢，或许终有一天会后悔冬天，因为为未来做准备的蚂蚁最终……舒适温暖。9在比赛的策划中，我们发现蚱蜢的下场可能不会那么糟糕。现在是蚂蚁复仇的时候了。

As we saw, a greedy algorithm works by taking the best decision at each step—which may not be the best decision overall. It has a somehow opportunistic nature or carpe diem feeling to it. Unfortunately, as Aesop’s fable tells us, a grasshopper living for the day may yet live to regret the winter, when the ant, who is preparing for the future, ends up cozy and warm.9 In the planning of tournaments, we found that the grasshopper may not end up so badly. Now it is time for the ant’s revenge.

在第一章中，我们讨论了通过枚举所有可能的路径来寻找网格上两点之间的最短路径的不可行性。我们发现，由于路径数量急剧增加，这在实践中根本无法实现。现在，凭借我们对图的了解，我们将发现一种方法。事实上，我们将把这个问题提升到一个更高的层次。我们不再寻找网格上的最短路径（网格具有某种良好的几何形状，并且所有点之间的距离都相等），而是允许任何几何形状，甚至可以添加不同的点间距离。

In chapter 1, we discussed the infeasibility of trying to find the shortest path between two points on a grid by enumerating all the possible paths. We saw that this is impossible to do in practice because the number of paths increases tremendously. Now with our knowledge of graphs, we will see that there is a way. In fact, we’ll take the problem up a notch. Instead of looking for the shortest path on a grid, which has a kind of nice geometry and on which all distances between points are equal, we will allow any geometric shape and even add different distances between points.

为此，我们将创建一个图，其中的节点和边代表一幅地图，并希望找到地图上两个节点之间的最短路径。此外，我们将为每条边附加一个权重。权重可以是正数或零，对应于两个连接节点之间距离的度量。它可以是英里数的距离或小时数的行程时间；任何其他非负指标都可以。路径长度是路径上所有权重的总和；两个节点之间的最短路径是长度最短的路径。如果所有权重都等于 1，则路径长度等于路径上的边数。一旦我们允许权重具有其他值，情况就不再如此。

To do that, we’ll create a graph where we have nodes and edges representing a map, and want to find the shortest way between two nodes on the map. Moreover, we’ll attach a weight to each edge. The weight may be positive or zero, and will correspond to a measure of the distance between the two connected nodes. It may be distance in miles or travel time in hours; any other nonnegative metric will do. Then the path length is the sum of the weights along the path; the shortest path between two nodes is the path with the smallest length. If all weights are equal to one, then the path length is equal to the number of edges on the path. Once we allow weights to have other values, this is no longer true.

在下图中，我们有六个节点，由九条具有不同权重的边连接，并希望找到从节点 A 到 F 的最短路径。

In the following graph, we have six nodes connected by nine edges with varying weights, and want to find the shortest path to travel from nodes A to F.

如果我们采用贪婪启发式算法，我们会从节点 A 出发到 C，那么最佳选择是前往节点 E，然后从那里前往节点 F。路径 A、C、E 和 F 的总长度为 8，但这并非最佳路径。最佳路径是从 A 到 C 再到 D，然后到 F，总长度为 6。因此，贪婪启发式算法不起作用。与锦标赛规划算法不同，它无法保证其相对于实际最短路径的最差性能。然而，与锦标赛规划算法不同，存在有效的算法来寻找最短路径，因此实际上根本没有理由使用贪婪启发式算法。

If we adopt a greedy heuristic, we’ll start by going from node A to C, then the best choice is to go to node E, and from there we make our way to node F. The total length of the path A, C, E, and F is eight, which is not, however, the best path. The best path is to go from A to C to D, and then to F, for a total length of six. So the greedy heuristic does not work, and in contrast to tournament planning, there are no guarantees as to its worst performance in relation to the actual shortest path. Nevertheless, and again in contrast to tournament planning, there exist efficient algorithms for finding the shortest paths so in fact there is no reason to use the greedy heuristic at all.

1956年，年轻的荷兰计算机科学家埃兹格·迪杰斯特拉（Edsger Dijkstra）和未婚妻在阿姆斯特丹购物。他们逛累了，便在一家咖啡馆的露台上坐下来喝咖啡。迪杰斯特拉一边喝咖啡，一边思考如何找到从一个城市到另一个城市的最佳路线。他只用了20分钟就设计出了解决方案，尽管该算法花了三年时间才得以发表。迪杰斯特拉的职业生涯辉煌，但令他惊讶的是，这20分钟的发明却成为了他成名的基石。10

In 1956, a young Dutch computer scientist, Edsger Dijkstra, was shopping in Amsterdam with his fiancée. Having got tired, they sat down at a café terrace to drink a cup of coffee, where Dijkstra thought about the problem of finding the best way to go from one city to another. He designed the solution in 20 minutes, although the algorithm took some time, three years, to get published. Dijkstra led an illustrious career, yet this 20-minute invention remained, to his amazement, a cornerstone of his fame.10

那么这个算法是如何进行的呢？我们想要找到图中从一个节点到所有其他节点的最短路径。该算法使用了一种叫做松弛（relaxation ）的思想：我们为想要找到的值（这里指的是距离）赋值。一开始，我们的估计值是最糟糕的。然后，随着算法的进展，我们可以将这些估计值从一开始的极差逐渐松弛到越来越好，直到得到正确的值。

So how does the algorithm go? We want to find the shortest paths from one node to all other nodes in a graph. The algorithm uses an idea called relaxation: we assign estimates for the values we want to find (here, distances). In the beginning, our estimates are the worst possible. Then as the algorithm progresses, we are able to relax these estimates from the extremely bad ones we started with to progressively better and better ones, until we arrive at the correct values.

在 Dijkstra 算法中，松弛过程如下。我们首先为所有节点到起始节点的距离分配一个最差的可能值：我们将距离设为无穷大；显然不可能有比这更差的了！在下图中，我们将最短路径的初始估计值及其路径上前一个节点分别放在每个节点的上方或下方。对于 A 节点，我们有，因为从 A 到 A 的距离为零，并且没有到 A 的前一个节点。对于所有其他节点，我们有，因为距离为无穷大，并且我们不知道到它们的最短路径。

In Dijkstra’s algorithm, relaxation proceeds as follows. We begin by assigning the worst possible value for the distances of all nodes from our starting node: we set the distance to infinity; clearly there cannot be anything worse than that! In the following figure, we have placed the initial estimate for the shortest path and previous node in that path above or below each node. For the A node, we have because the distance from A to A is zero and there is no previous node to A. For all other nodes, we have because the distance is infinity and we have no idea about the shortest path to them.

我们选取目前距离 A 最短的节点。这个节点就是 A 本身。它是我们的当前节点，因此我们将其标记为灰色。

We take the node with the shortest distance from A thus far. This is A itself. That is our current node, so we mark it gray.

从 A 出发，我们可以检查到其邻居 B 和 C 的最短路径的估算值。最初我们将它们设置为无穷大，但实际上现在我们发现，从 A 到 B 的代价是 3，而从 A 到 C 的代价是 1。我们更新这些估算值，并指出这些估算值是通过 A 实现的；我们在 B 上方写 3/A，在 C 下方写 1/A。在算法的剩余部分，我们完成了对节点 A 的处理。我们相应地更新图形，将 A 标记为黑色。我们移至当前估算值最优的未访问节点。即节点 C。

From A we can check the estimates for the shortest paths to its neighbors, B and C. Initially we had set them at infinity, but in fact now we find out that we can get to B from A at a cost of 3 and we can get to C from A at a cost of 1. We update these estimates and also indicate that the estimates are through A; we write 3/A above B and 1/A below C. We are done with node A for the rest of the algorithm. We update the figure accordingly, marking A black. We move to the unvisited node with the best current estimate. That is node C.

从节点 C 开始，我们检查到其邻居节点 D 和 E 的最短路径估计值。它们位于无穷远处，但是现在我们看到我们可以通过 C 到达每个节点。从 A 到 D 经过 C 的路径总长度为 5，因此我们在 D 上方写上 5/C。从 A 到 E 经过 C 的路径总长度为 3，因此我们在 E 下方写上 3/C。我们已经完成了节点 C 的访问，因此将其标记为黑色，并移至当前估值最佳的未访问节点。节点 B 和 E 的估值均为 3，两者皆为最佳。我们可以任选其一。我们选择 B。

From node C, we check the estimates of the shortest paths to its neighbors, D and E. They were at infinity, but now we see that we can get to each one of them through C. The path from A to D through C has a total length of 5, so we write 5/C above D. The path from A to E through C has a total length of 3, so we write 3/C below E. We are done with node C so we mark it black and move to the unvisited node with the best current estimate. Both nodes B and E have an equally good estimate of 3. We can pick either. Let us pick B.

我们以相同的方式工作。从节点 B 开始，我们检查到其邻居节点 D 和 F 的最短路径的估计值。我们已经从 C 得到了 D 的长度估计值 5；这比我们得到的长度 6 要好。来自 B。因此，我们让对 D 的估计保持不变。对 F 的当前估计是无穷大，因此我们将其更新为 9，来自 B。我们将 B 标记为已访问，并移至具有最佳当前估计的未访问节点。即节点 E。

We work in the same way. From node B, we check the estimates of the shortest paths to its neighbors, D and F. We already have an estimate of length 5 for D, coming from C; that is better than the length 6 that we would get coming from B. So we let the estimate to D remain unchanged. The current estimate to F is infinite so we update it to 9, coming from B. We mark B as visited and move to the unvisited node with the best current estimate. That is node E.

E 的邻居是 F。从 E 到 F 的路径长度为 8，这比我们之前找到的通过 B 的路径要好。我们更新该路径，将 E 标记为已访问，并移至当前估计值最优的未访问节点，即节点 D。

E has F as a neighbor. The path to F from E has length of 8, which is better than the path we had found through B. We update the path, mark E as visited, and move to the unvisited node with the best current estimate, node D.

D 的邻居是 F，我们找到了一条从 E 到 F 的路径，长度为 8。由于我们可以通过 D 到达 F，总长度为 6，因此我们更新该路径。和之前一样，我们移动到当前估计值最优的未访问节点——实际上也是我们唯一未访问的节点 F。

D has F as a neighbor, to which we have found a path coming from E with length 8. As we can get to F through D with a total length of 6, we update that path. As before, we move to the unvisited node with the best current estimate—actually our only unvisited node, F.

从节点 F 开始，我们检查是否应该更新对其邻居节点 E 的估计。当前到 E 的路径长度为 3，而经过 F 的路径成本为 10。我们让 E 保持不变。访问 F 不会带来任何影响，但我们事先不可能知道这一点。由于我们已经访问了所有节点，算法结束。

From node F we check whether we should update our estimate for its neighbor, node E. The current path to E has a length of 3, while the path through F would have a cost of 10. We let E remain unchanged. Visiting F did not make any difference, but we could not have known that beforehand. As we have visited all nodes, the algorithm finishes.

在算法过程中，我们记录了最短路径上每个节点的路径长度和前一个节点。这样做是为了在算法完成后，如果想要找到从 A 到图中任何其他节点（例如 F）的最短路径，我们就从末端开始，一直走到起点。我们读到它的前身：D。我们得到 D 的前身是 C，然后 C 的前身是 A。从 A 到 F 的最短路径是 A、C、D 和 F，总长度为 6，正如我们在讨论开始时提到的那样。

When we were going through the algorithm, we were recording path lengths and the predecessor of each node along the shortest path. We did that so that if after finishing the algorithm we want to find the shortest path from A to any other node in the graph—for example, F—we start from the end and make our way to the start. We read its predecessor: D. We get the predecessor of D, which is C, and then the predecessor of C, which is A. The shortest path from A to F is A, C, D, and F with a total length of six, as we had mentioned way back at the start of our discussion.

最后，Dijkstra 算法找到了图中从起始节点到所有其他节点的所有最短路径。该算法非常高效，因为它的复杂度为，其中m是图中的边数，n是节点数。该算法分为以下几个步骤：

At the end, Dijkstra’s algorithm found all the shortest paths from the starting node to all other nodes in the graph. The algorithm is efficient, as its complexity is , where m is the number of edges in the graph and n is the number of nodes. Here is the algorithm as a set of steps:

1. 1. 为除起始节点之外的所有节点分配等于无穷大的距离；为起始节点分配等于零的距离。
2. 1. Assign a distance equal to infinity to all nodes except for the starting node; assign a distance equal to zero for the starting node.
3. 2. 找到距离最小的未访问节点。该节点将成为当前节点。如果没有未访问节点，则停止。
4. 2. Find the unvisited node with the minimum distance. This will be our current node. If there is no unvisited node, stop.
5. 3. 检查当前节点的所有邻居节点。如果它们的距离大于从当前节点到邻居节点的距离，则放宽距离并更新到邻居节点的路径。转到步骤 2。
6. 3. Examine all neighbors of the current node. If their distance is greater than the distance we would get passing from the current node before arriving at the neighbor, we relax the distance and update the path going to the neighbor. Go to step 2.

如果我们只对到特定节点的最短路径感兴趣，我们可以在步骤 2 中选择该节点进行访问时停止。一旦我们这样做，我们就已经找到了到达它的最短路径，并且它在算法的其余执行过程中不会改变。

If we are only interested in the shortest path to a particular node, we can stop when we pick it to visit in step 2. Once we do that, we have already found the shortest path to it, and it will not change in the rest of the algorithm’s execution.

只要图没有负权重，我们就可以在任何图中使用 Dijkstra 算法，无论图是否有向，即使它包含循环。如果边表示节点之间的某种奖励和惩罚，则可能会发生这种情况。好消息是，在存在负权重的情况下，我们可以使用其他有效的算法，但这突显出算法在适用性方面可能有特殊要求。当我们尝试寻找一种算法来解决我们的问题时，我们应该检查我们的问题是否满足算法的要求。否则算法将不起作用；但请注意，算法不能告诉我们它不起作用。如果我们在计算机上实现该算法，它仍然会执行其步骤，即使这样做没有意义。它会给出一个无意义的答案。我们必须确保我们使用正确的工具来完成正确的工作。

We can use Dijkstra’s algorithm in any graph, directed or not, even if it contains cycles, provided that it does not have negative weights. This might happen if the edges represent some kind of rewards and penalties between nodes. The good news is that there are other efficient algorithms that we can use in the presence of negative weights, but this highlights that algorithms may have particular requirements in their applicability. When we try to find an algorithm to solve our problem, we should check that our problem meets the requirements of the algorithm. Otherwise the algorithm will not work; but note that an algorithm cannot tell us that it does not work. If we implement the algorithm on a computer, it will still execute its steps even if it does not make sense to do so. It will produce an answer that will be nonsense. It is up to us to make sure that we are using the right tool for the right job.

举一个极端的例子，想象一下，如果一个图不仅权重为负，而且还包含一个边数和为负的环，会发生什么：一个负环。那么，任何算法都无法找到图中的最短路径，因为它们根本不存在。如果我们有一个负环，我们可以绕着它的边一圈一圈地走，每次路径的长度都会减少。我们可以一直这样下去，沿着这个环的路径最终会变成负数。无穷大。计算机科学家和程序员对在程序中放入一些对程序来说毫无意义的东西有一个说法：垃圾进，垃圾出。人类有责任找出垃圾，并知道何时使用什么。大学算法课程的一个重要部分就是教会崭露头角的计算机科学家何时使用什么。

For an extreme example, think of what would happen with a graph that not only has negative weights but also a cycle where the sum of the edges is negative: a negative cycle. Then no algorithm would find the shortest paths in the graph because they do not exist. If we have a negative cycle, we can go round and round its edges, and every time the length of the path will be reduced. We can continue forever, and the path along the cycle will get to negative infinity. Computer scientists and programmers have a name for when we put something in a program that does not make sense for it: garbage in, garbage out. It is up to humans to ferret out the garbage and know what to use when. An important part of algorithm courses in universities is exactly to teach budding computer scientists what to use when.

大海捞针

A Needle in a Haystack

最简单的搜索方法就像我们俗话说的大海捞针。如果我们想在一组对象中找到某个东西，而它们之间完全没有结构，那么我们唯一能做的就是一个接一个地检查，直到找到我们要找的东西，或者在用尽所有东西之后仍然找不到为止。

The simplest way to search is what we do to find the proverbial needle in a haystack. If we want to find something in a group of objects and there is absolutely no structure in them, then the only thing we can do is to check one item after the other until we either find the item we are looking for or fail to find it after exhausting all items.

如果你有一副牌，并且正在寻找其中的某一张，你可以从牌堆顶部开始取牌，直到找到你要找的那张或牌用完为止。或者，你也可以从牌堆底部开始一张一张地取牌。你甚至可以从牌堆中的随机位置取牌。原理是一样的。

If you have a deck of cards and are looking for a particular one in them, you can start taking off the cards from the top of the deck until you find the one you are looking for or run out of cards. Alternatively, you can start taking off the cards one by one from the bottom of the deck. You can even take off cards from random positions in the deck. The principle is the same.

通常我们不处理计算机中的物理对象，而是处理它们的数字表示。在计算机上表示数据组的一种常见方式是采用列表的形式。列表是一种数据结构，它以这样的方式包含一组事物，即我们可以从一个项目找到下一个项目。我们通常可以将列表想象为包含链接的项目，其中一个项目指向下一个项目，直到最后，最后一项指向任何内容。这个比喻与事实相差不远，因为在计算机内部，使用内存位置来存储项目。在链接列表中，每个项目包含两样东西：它的有效载荷数据以及列表中下一个项目的内存位置。内存中保存另一个位置的内存位置的位置称为指针。因此，在链表中，每个元素都包含指向下一个元素的指针。列表的第一个项目称为其头。列表中的项目也称为节点。最后一个节点不指向任何地方；我们说它指向null：计算机上的虚无。

Usually we do not deal with physical objects in computers but rather digital representations of them. A common way to represent groups of data on a computer is in the form of a list. A list is a data structure that contains a group of things in such a way that from one item we can find the next one. We can usually think of the list as containing linked items, where one item points to the next one, until the end, where the last item points to nothing. The metaphor is not far from the truth because internally the computer uses memory locations to store items. In a linked list, each item contains two things: its payload data and the memory location of the next item on the list. A place in memory that holds the memory location of another place in memory is called a pointer. Therefore in a linked list, each element contains a pointer to the next element. The first item of a list is called its head. The items in a list are also called nodes. The last node does not point to anywhere; we say that it points to null: nothingness on a computer.

列表是一系列元素的序列，但该序列不一定按照某些特定标准排序。例如，以下列表包含字母表中的一些字母：

A list is a sequence of items, but it is not necessary that the sequence is ordered using some specific criterion. For example, the following is a list containing some letters from the alphabet:

如果我们有一个无序列表，则查找其中的项目的算法如下：

If we have an unordered list, the algorithm for finding an item on it goes like this:

1. 1. 转到列表的开头。
2. 1. Go to the head of the list.
3. 2. 如果该物品是我们正在寻找的，则报告已找到并停止。
4. 2. If the item is the one we are looking for, report that it is found and stop.
5. 3. 转到列表中的下一个项目。
6. 3. Go to the next item on the list.
7. 4. 如果结果为空，则报告未找到搜索项并停止。否则，返回步骤 2。
8. 4. If we are at null, report that the search item was not found and stop. Otherwise, return to step 2.

这被称为线性搜索或顺序搜索。它本身并没有什么特别之处；它只是对“依次检查每个元素，直到找到我们想要的那个”这一想法的直接实现。实际上，该算法会让计算机从一个指针跳到另一个指针，直到找到我们要找的元素或为空。下面我们展示了搜索 E 或 X 时发生的情况：

This is called a linear or sequential search. There is nothing special about it; it is a straightforward implementation of the idea of examining each single thing in turn until we find the one we want. In reality, the algorithm makes the computer jump from pointer to pointer until it either reaches the item we are looking for or null. Below we show what is happening when we search for E or X:

如果我们在n 个项目中搜索，最好的情况是立即找到我们想要的项目，如果它位于列表的头部，则会发生这种情况。最糟糕的情况是该项目位于列表的末尾，或者根本不在列表中。那么我们必须遍历所有n 个项目。因此，顺序搜索的性能是。

If we search among n items, the best thing that can happen is to hit on the item we want immediately, which will occur if it is the head of the list. The worst thing that can happen is that the item is the last one on the list or not on the list at all. Then we must go through all n items. Therefore the performance of sequential search is .

如果列表中物品的顺序是随机的，我们就无法提高时间效率。回到一副牌的例子，你就能明白为什么会这样：如果牌组洗得正确，我们就无法提前知道牌会在哪里找到。

There is nothing we can do to improve on that time if the items appear on the list in a random sequence. Going back to a deck of cards, you can see why this is so: if the deck is properly shuffled, there is no way to know in advance where we’ll find our card.

有时人们会遇到麻烦。如果我们在一大堆纸里找一张，我们可能会厌倦去寻找。一张接一张。我们甚至会想，如果这张纸放在了最底下，那我们该有多倒霉！所以我们不再按顺序翻阅，而是偷看最底下。偷看最底下并没有错，但认为这样能提高快速完成搜索的几率就错了。如果这堆纸是随机的，那么我们想要的东西没有理由不在最上面、最下面，或者最中间。任何位置出现的可能性都一样大，所以从最上面开始，一直翻到最底下，和其他确保我们只查看一次每件物品的策略一样好。然而，如果我们按照特定的顺序搜索，通常比漫无目的地跳来跳去更容易记住我们看过的东西，这就是为什么我们更喜欢坚持顺序搜索。

Sometimes people have trouble with that. If we are looking for a paper among a large pile, we may tire of going one after the other. We may even think of how unlucky we would be should the paper turn out to be at the bottom of the pile! So we stop going through the pile in order and peek at the bottom. There is nothing wrong in peeking at the bottom, but it’s wrong to think that this improves our chances of finishing the search quickly. If the pile is random, then there is no reason why the sought-after item is not the first, last, or one right in the middle. Any position is equally likely, so starting from the top and making our way to the bottom of the pile is as good a strategy as any other that ensures we examine each item exactly once. It is usually simpler to keep track of what we looked at if we work in a specific order, however, than jumping around erratically, and that’s why we prefer to stick with a sequential search.

只要没有理由怀疑搜索项位于特定位置，上述方法就成立。但如果情况并非如此，情况就会发生变化，我们可以利用任何额外的信息来加快搜索速度。

All this holds as long as there is no reason to suspect that the search item is in a particular position. But if this is not true, then things change, and we can take advantage of any extra information we may have to speed up our search.

马太效应与搜索

The Matthew Effect and Search

你可能注意到，在凌乱的书桌上，有些东西会堆到最上面，而有些东西则会滑到最下面。当你终于把混乱之中，作者曾有过一段愉快的经历：在一堆深埋的东西里，发现了一些他以为早已丢失的东西。其他人可能也经历过这样的经历。我们倾向于把常用的东西放在身边；而那些不常用的东西，却越放越远，伸手可及。

You may have noticed that in an untidy desk, some things find their way to the top of the pile, while some others seem to slip to the bottom. When finally cleaning up the mess, the author has had the pleasant experience of discovering buried deep down in a heap things he believed were long lost. The experience has probably occurred to others as well. We tend to place things we use frequently close; things we have little use for slip further and further out of reach.

假设我们有一堆文件需要处理。这些文件没有任何顺序。我们翻阅着，找到我们需要的文件，处理完后，就把它放到了最上面，而不是原来的位置。然后我们继续处理其他事情。

Suppose we have a pile of documents on which we need to work. The documents are not ordered in any way. We go through the pile, searching for the document we need, processing it, and then placing it not where we found it but instead on the top of the pile. Then we go again with our business.

我们处理所有文档的频率可能并不一致。有些文档我们可能会反复查看，而有些文档我们可能很少访问。如果我们在处理完每份文档后都将其放在最上面，一段时间后我们会发现，最常用的文档会靠近顶部，而访问频率最低的文档则会移到底部。这对我们来说很方便，因为我们可以减少查找常用文档的时间，从而减少总体时间。

It may happen that we do not work with the same frequency on all documents. We may return to some of them again and again, while we may only rarely visit others. If we continue placing every document on the top of the pile after working on it, after some time we’ll find out that the most popular documents will be near the top, while the ones we accessed the least often will have moved toward the bottom. This is convenient for us because we spend less time locating the frequently used documents and thus less time overall.

这体现了一种通用的搜索策略：我们会反复搜索相同的商品，有些商品比其他商品更受欢迎。找到一件商品后，就把它放在前面，这样下次查找时就能更快地找到它。

This suggests a general searching strategy, where we search for the same items repeatedly, and some items are more popular than others. After finding an item, bring it forward so that we’ll be able to find it faster the next time we will look for it.

这种策略的适用性如何？这取决于我们观察到这种受欢迎程度差异的频率。事实证明，这种情况经常发生。我们都知道“富人越来越富，穷人越来越穷”这句话。这不仅仅关乎富人和穷人。同一件事在不同活动领域以令人眼花缭乱的方式出现。这种现象有一个名字，叫做马太效应，取自《马太福音》（25:29）中的以下经文：“因为凡有的，还要加给他，叫他有余；没有的，连他所有的也要夺过来。”

How applicable would such a strategy be? It depends on how often we observe such differences in popularity. It turns out that they happen a lot. We know the saying “the rich get richer, and the poor get poorer.” It is not just about rich and poor people. The same thing appears to a bewildering array of aspects in different fields of activity. The phenomenon has a name, the Matthew effect, after the following verse in the Gospel of Matthew (25:29): “For unto every one that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath.”

这节经文谈论的是物质财富，那么我们不妨先来思考一下财富。假设你有一个大型体育场，可容纳八万人。你测量了体育场内所有人的平均身高，结果可能是1.70米（5英尺7英寸）左右。想象一下，你随机从体育场中抽出一个人，然后把世界上最高的人放进去。平均身高会有所不同吗？即使最高的人身高3米（历史上从未有过这样的身高记录），平均身高仍会保持原来的值——与之前的平均值的差异不到十分之一毫米。

The verse talks about material goods, so let’s think about wealth for a minute. Suppose you have a large stadium, capable of holding 80,000 people. You are able to measure the average height of the people in the stadium. Your result may be something around 1.70 meters (5 feet, 7 inches). Imagine that you take out somebody randomly from the stadium and put in the tallest person in the world. Will the average height differ? Even if the tallest person is 3 meters tall (no such height has ever been recorded), the average height would remain stuck at its previous value—the difference with the previous average being less than a tenth of a millimeter.

现在想象一下，你测量的不是平均身高，而是平均财富。你的8万人的平均财富可能是100万美元（我们假设这是一个富裕的群体）。现在你又换成某人和世界首富一起。这个人的财富可能高达1000亿美元。这会有什么不同吗？是的，会的，而且是巨大的不同。平均财富将从100万美元增加到2249987.5美元，增长一倍以上。我们知道，世界各地的财富分配并不均等，但我们可能没有意识到这种分配有多么不均衡。这种不均衡远比身高等自然指标的分配更为严重。

Imagine now that instead of measuring the average height, you measure the average wealth. The average wealth of your 80,000 people could be $1 million (we are assuming a wealthy cohort). Now you substitute again somebody inside with the richest person in the world. That person could have a wealth of $100 billion. Would this make a difference? Yes, it would—and a big one. The average would increase from $1 million to $2,249,987.5, or more than double. We are aware that wealth is not distributed equally around the world, but we may not be aware of how unequal the distribution is. It is much more unequal than a distribution of natural measures like height.

同样的禀赋差异也存在于许多其他环境中。有很多演员你闻所未闻。也有一些明星出演过多部电影，收入数百万美元。“马太效应”一词由社会学家罗伯特·K·默顿于1968年提出，他观察到，即使贡献相似，著名科学家也会比不太知名的同事获得更多的赞誉。科学家越出名，他们就越出名。

The same difference in endowments occurs in many other settings. There are many actors you have never heard of. And there are a few stars who have appeared in many movies, earning millions of dollars. The term “Matthew effect” was coined by the sociologist Robert K. Merton in 1968, when he observed that famous scientists get more credit for their work over their lesser-known colleagues, even if their contributions are similar. The more famous scientists are, the more famous they will get.

一种语言中的词汇遵循着同样的模式：有些词汇比其他词汇更受欢迎。这种显著的不平等现象体现在诸多领域，包括城市规模（特大城市比普通城市大很多倍），以及网站的数量、链接和受欢迎程度（大多数网站只有偶尔的访客，而有些网站则能带来数百万的流量）。这种不平等分配的普遍性，即少数群体获得不成比例的资源，在过去一直是备受关注的研究领域。几年来，研究人员正在研究此类现象出现的原因和规律。1

Words in a language follow the same pattern: some of them are much more popular than others. The list of domains that are characterized by such jarring inequalities includes the size of cities (megacities are many times larger than the average city) and number, links, and popularity of web sites (most sites are honored only by the occasional visitor, while others rake in millions). The prevalence of such unequal distributions, where a few elements of a population obtain a disproportionate amount of resources, has been a rich field of inquiry over the last few years. Researchers are looking into the reasons and laws that underlie the emergence of such phenomena.1

我们搜索的项目可能存在这样的流行度差异。那么，一个利用搜索项目不同流行度的搜索算法，其工作原理类似于将我们找到的每个文档放在最上面：

It is possible that the items in which we are searching exhibit such differences in popularity. Then a search algorithm that will take advantage of the varying popularity of the search items can work much like putting each document that we find at the top of the pile:

1. 1. 使用顺序搜索来搜索该项目。
2. 1. Search for the item using a sequential search.
3. 2. 如果找到了该项，则报告找到，并将其放在列表的最前面，然后停止。
4. 2. If the item is found, report that it is found, put it at the front of the list, at its head, and stop.
5. 3. 否则，报告未找到该项目并停止。
6. 3. Otherwise, report that the item was not found and stop.

下图中，在列表中找到 E 会将其置于最前面：

In the following figure, finding E on the list will bring it to the front:

这种移至前端算法的一个可能的缺点是，它甚至会将我们很少搜索的项目提升到前端。这没错，但如果该项目没有流行，当我们搜索其他项目时，它会逐渐移到列表末尾，因为这些项目会移到前面。不过，我们可以采取一种不那么极端的策略来处理这种情况。与其将找到的每个项目都移到前面，不如将其向前移动一位。这被称为转置法：

A possible criticism of this move-to-front algorithm is that it will promote to the front even an item that we only rarely search for. That is true, but if the item is not popular, it will gradually move toward the end of the list as we search for other items because these items will move to the front. We can take care of the situation, however, by adopting a less extreme strategy. Instead of moving each item we find bang to the front, we can move it just one position forward. This is called the transposition method:

1. 1. 使用顺序搜索来搜索该项目。
2. 1. Search for the item using a sequential search.
3. 2. 如果找到了该物品，则报告找到，并将其与前一个物品交换（如果不是第一个），然后停止。
4. 2. If the item is found, report that it is found, exchange it with the previous one (if it is not the first one), and stop.
5. 3. 否则，报告未找到该项目并停止。
6. 3. Otherwise, report that the item was not found and stop.

这样，受欢迎的商品就会逐渐移到前面，而不太受欢迎的商品则会移到后面，而不会出现突然的动荡。

In this way, items that are popular will gradually make their way to the front, and less popular items will move to the back, without sudden upheavals.

移至前端和转置方法都是自组织搜索的例子；其名称来源于因为项目列表是随着我们的搜索而组织的，并且会反映所搜索项目的受欢迎程度。根据项目受欢迎程度的范围，节省的成本可能非常可观。虽然顺序搜索的预期性能为，但使用移至前端方法的自组织搜索可以达到的性能为。如果我们有大约一百万个项目，那么这就是一百万和五万个之间的差异。转置方法可以获得更好的结果，但需要更多时间才能实现。这是因为这两种方法都需要一个“预热期”，在此期间热门项目会自行出现并移到前面。在移至前端方法中，预热时间很短；在转置方法中，预热时间需要更长，但我们会得到更好的结果。2

Both the move-to-front and transposition methods are examples of a self-organizing search; the name comes because the list of items is organized as we go with our searches and will reflect the popularity of the searched items. Depending on how the popularity ranges among items, the savings can be significant. While with a sequential search we can expect a performance of , a self-organizing search with the move-to-front method can attain a performance of . If we have about a million items, this is the difference between 1 million and about 50,000. The transposition method can have even better results, but it requires more time to achieve them. That’s because both methods require a “warm-up period” in which popular items will show themselves up and make their way to the front. In the move-to-front method, the warm-up is short; in the transposition method, the warm-up takes longer, but then we get better results.2

开普勒、汽车和秘书

Kepler, Cars, and Secretaries

著名天文学家约翰尼斯·开普勒（1571-1630）于1611年因霍乱去世后，便着手再婚。他为人处事严谨，从不听天由命。在一封写给斯特拉伦多夫男爵的长信中，他描述了自己选择的流程。他计划面试11位可能的新娘，然后再做决定。第五位候选人对他很有吸引力，但他的朋友反对她身份卑微，最终说服了他。他们建议他重新考虑第四位候选人。但她拒绝了他。最终，在考察了所有11位候选人之后，开普勒最终娶了第五位：24岁的苏珊娜·罗伊廷格。

After the celebrated astronomer Johannes Kepler (1571–1630) lost his wife to cholera in 1611, he set out to remarry. A methodical man, he did not leave things to chance. In a long letter to a Baron Strahlendorf, he describes the process he followed. He planned to interview 11 possible brides before making his decision. He was strongly attracted to the fifth candidate, but was swayed against her by his friends, who objected to her lowly status. They advised him to reconsider the fourth candidate instead. But then he was turned down by her. In the end, after examining all 11 candidates, Kepler did marry the fifth one: 24-year-old Susanna Reuttinger.

这个小故事是一个延伸的搜索例子；开普勒在众多可能的候选者中寻找理想的匹配对象。然而，这个过程存在一个他一开始可能并未意识到的缺陷：一旦他拒绝了某个可能的匹配对象，就可能无法再回到原来的匹配对象。

This little story is a stretched example of a search; Kepler was searching for an ideal match, among a pool of possible candidates. Yet there was a kink in the process that he was probably not aware of when he started: it might not be possible to go back to a possible match after he had rejected it.

我们可以用更现代的术语重新表述这个问题，即寻找最佳方法来决定购买哪辆车。我们事先决定要去一定数量的汽车经销店。而且，我们的自尊心不允许我们在离开后再回到一家汽车经销店。如果我们拒绝了某辆车，面子至关重要，这样我们就不能回去说我们改变了主意。或者，也许在我们离开后，其他人走进来买了这辆车。无论如何，我们都必须在每家经销店做出最终决定，是买下这辆车，还是放弃，不再回来。

We can recast the problem in more contemporary terms, as looking for the best way to decide which car to buy. We have decided beforehand that we will visit a certain number of car dealerships. Also, our amour propre will not allow us to return to a car dealership after we have walked away from it. If we have declined a car, saving face is paramount, so that we cannot go back and say that we changed our mind. Or perhaps somebody else walked in and bought the car after we left. Be it as it may, we have to make a final decision at each dealership, to buy the car or let go, and not come back.

这是一个最优停止问题的例子。我们必须采取行动，同时努力最大化回报或最小化成本。在我们的例子中，我们想要决定购买一辆汽车，并且这个决定最终能买到最好的车。如果我们决定得太早，我们可能会买到一辆比我们还没见过的车更差的车。如果我们决定购买一辆比我们还没见过的车更差的车。如果我们决定购买一辆比我们还没见过的车更差的车，那么最终的结果是，我们最终会买 ...太晚了，我们可能会懊恼地发现，我们竟然看到了最好的车，却错过了。什么时候才是停下来做决定的最佳时机？

This is an instance of an optimal stopping problem. We have to take an action, while trying to maximize a reward or minimize a cost. In our example, we want to decide to buy the car, when this decision will result in the best car we can buy. If we decide too early, we may settle on a car that is worse than a car we have not seen yet. If we decide too late, we may discover to our chagrin that we saw, but missed, the best car. When is the optimal time to stop and make a decision?

同样的问题，通常被描述得更冷酷无情，就像秘书问题一样。你想从众多候选人中挑选一位秘书。你可以逐一面试候选人，但每次面试结束后，你都必须决定是否聘用。如果你拒绝了一位候选人，之后就不能改变主意再录用（这位候选人可能太优秀了，会被其他人抢走）。那么，你该如何挑选这位候选人呢？

The same issue is usually described in a more callous way as the secretary problem. You want to select a secretary from a pool of candidates. You can interview the candidates one by one. You must make a decision to hire or not at the end of each interview, however. If you reject a candidate, you cannot later change your mind and make an offer (the candidate might be too good and thus be snapped by somebody else). How will you pick the candidate?

答案出奇地简单。你筛选前 37% 的候选集，全部淘汰，但保留其中最好的一个作为基准。37 这个数字看起来很神奇，因为它的出现是因为，其中e是欧拉常数，约等于 2.7182（我们在第一章中见过欧拉常数）。然后你筛选剩余的候选集。你选择的第一个比基准集更优的候选集作为基准集。用算法的形式来表达，假设你有n 个候选集：

There is a surprisingly simple answer. You go through the first 37 percent of the candidates, rejecting them all, but keeping a tab on the best one among them as your benchmark. The number 37, which seems magical, occurs because , where e is Euler’s number, approximately equal to 2.7182 (we saw Euler’s number in chapter 1). Then you go through the rest of the candidates. You stop at the first of the rest that is better than your benchmark. That will be your pick. In algorithmic form, if you have n candidates:

1. 1. 计算，找出n个候选人中的37%。
2. 1. Calculate , to find the 37 percent of the n candidates.
3. 2. 审查并淘汰第一批候选人。你将使用其中最好的一个作为基准。
4. 2. Examine and reject the first candidates. You will use the best one among them as a benchmark.
5. 3. 继续测试剩余的候选集。选出第一个比基准更好的，然后停止。
6. 3. Continue with the rest of candidates. Pick the first one that is better than your benchmark, and stop.

算法并非总能找到最佳候选；毕竟，总体而言，最佳候选可能是你在前 37% 的案例中确定的基准候选，而你最终将其排除。可以证明，它能在 37%（同样，）的所有案例中找到最佳候选；而且，没有其他方法能够在更多案例中找到最佳候选。换句话说，算法是你所能做到的最好的：尽管它可能在 63% 的案例中无法给出最佳候选，但你选择遵循的任何其他策略在更多案例中都会失败。

The algorithm will not always find the best candidate; after all, the best candidate overall may be the benchmark candidate you identified in the first 37 percent, and that you have rejected. It can be proved that it will find the best candidate in 37 percent (again, ) of all cases; moreover, there is no other method that will manage to find the best candidate in more cases. In other words, the algorithm is the best you can do: although it may fail to give you the best candidate in 63 percent of the cases, any other strategy you may decide to follow will fail in more cases than that.

回到汽车的话题，假设我们决定去10家汽车经销商。我们应该去前四家，并记下这四家给出的最佳报价，但不要购买。然后，我们开始去剩下的六家经销商，并选择第一家报价比我们记下的报价更优的经销商购买（之后我们跳过剩下的）。我们可能会发现，这六家经销商给出的报价都比我们之前去的四家（但没买）的报价更差。但没有其他策略能让我们更有可能获得最佳交易。

Going back to cars, suppose we decide to visit 10 car dealerships. We should visit the first four and take note of the best offer by these four, without buying. Then we start visiting the remaining six dealerships and we’ll buy from the first dealership that gives us an offer better than the one we noted down (we’ll then skip the rest). We may discover that all six dealerships make worse offers then the first four that we visited without buying. But no other strategy can give us better odds of getting the best deal.

我们假设我们想要找到最好的候选人，并且不会满足于更低的条件。但如果我们实际上可以接受更低的条件呢？这意味着即使理想情况下我们想要最好的秘书或汽车，我们也可以接受其他选择，尽管我们可能不太满意，但即使选择了最好的，也不会那么满意。如果我们这样构建问题，那么进行选择的最佳方法是使用与上述相同的算法，但检查并舍弃候选者的平方根 。如果我们这样做，我们做出最佳选择的概率会随着候选者数量的增加而增加：随着n 的增加，我们选择最佳的概率会趋近于 1（即 100%）。3

We have assumed that we want to find the best possible candidate and will settle for nothing less. But what if we can in fact settle for something less? That means that even though ideally we would want the best secretary or car, we can make do with another choice, with which we may be happy, although not as happy had we picked up the best. If we frame the problem like that, then the best way to make our selection is to use the same algorithm as above, but examining and discarding the square root, , of the candidates. If we do that, the probability that we will make the best choice increases with the number of candidates: as n increases, the probability that we’ll pick the best goes to 1 (that is, 100 percent).3

二分查找

Binary Search

我们考虑了针对不同场景的不同搜索方法。所有这些方法的共同点在于，我们检查的项目并非按照特定顺序给出；在自组织搜索中，我们最多会根据热门程度逐渐排序。但如果项目一开始就已排序，情况就会完全不同。

We have considered different ways to search, corresponding to different scenarios. A common thread in all these was that the items that we examine are not given to us in any specific order; at best, we order them gradually by popularity in a self-organized search. The situation changes completely if the items are ordered in the first place.

假设我们有一堆文件夹，每个文件夹都用一个数字标识。这些文档根据其标识符从小到大排序（数字不必连续）。如果我们有这样一堆文档，并且正在寻找具有特定标识符的文档，那么从第一个文档开始一直到最后一个文档，直到我们找到了我们要找的那份。更好的策略是直接找到这堆文件的中间部分。然后，我们将中间文件上的编号标识符与我们要找的文件编号进行比较。结果可能有三种：

Let’s say we have a pile of folders, each one of which is identified by a number. The documents in the pile are ordered according to their identifier, from the lowest to highest number (there is no need for the numbers to be consecutive). If we have such a pile and are looking for a document with a particular identifier, it is foolish to start from the first document and make our way to the last until we find the one we are looking for. A much better strategy is to go straight to the middle of the pile. Then we compare the number identifier on the document in the middle to the number of the document that we are looking for. There are three possible outcomes:

1. 1. 如果幸运的话，我们可能正好找到了我们想要的文档。好了，我们的搜索结束了。
2. 1. If we are lucky, we may have landed exactly on the document that we want. We are done; our search is over.
3. 2. 我们要查找的文档的标识符大于我们手中文档的标识符。这样我们就可以确定可以丢弃手头的文档以及所有前面的文档。由于它们是有序的，所以它们的标识符都会比较小。我们没能达到目标。
4. 2. The identifier of the document we are looking for is greater than the identifier of the document we have in our hands. Then we know for sure that we can discard the document at hand as well as all preceding documents. As they are ordered, they will all have smaller identifiers. We have undershot our target.
5. 3. 相反的情况发生了：我们要查找的文档的标识符小于我们手中文档的标识符。那么我们可以放心地丢弃手头的文档以及它之后的所有文档。我们已经超出了目标。
6. 3. The opposite happens: the identifier of the document we are looking for is smaller than the identifier of the document we have in our hands. Then we can safely discard the document at hand as well as all the documents that come after it. We have overshot our target.

无论最后两种结果如何，我们现在剩下的文件堆最多只有原来文件的一半。如果我们从奇数个文档开始，比如n 个，那么将n 个文档从中间拆分成两部分，每部分都有项目（在拆分过程中丢弃小数部分）：

In either of the last two outcomes, we are now left with a pile that is at most half the original one. If we start with an odd number of documents, say n, splitting n documents in the middle gives us two parts, each with items (discarding the fractional part in the division):

如果项目数量为偶数，则拆分它们将得到两部分，一部分包含项目，另一部分包含项目：

With an even number of items, splitting them will give us two parts, one with items and another one with items:

我们仍然没有找到想要的东西，但比以前好多了；现在要检查的项目少了很多。我们确实找到了。我们检查了剩余项目的中间文档，然后重复了这个过程。

We have still not found what we were looking for, but we are much better than before; we have much fewer items to go through now. And so we do. We check the middle document of the remaining items and repeat the procedure.

在下一页的图中，您可以看到针对 16 个项目的流程如何展开，其中我们正在寻找项目 135。我们用灰色标出我们搜索的边界和中间项目。

In the figure on the following page, you can see how the process evolves for 16 items, among which we are looking for item 135. We mark out the boundaries inside which we search and the middle item with gray.

一开始，我们的搜索范围是所有项目的集合。我们找到中间的项目，发现它是 384。它大于 135，所以我们丢弃它，以及它右边的所有项目。我们取剩余项目的中间值，结果是 72。它小于 135，所以我们丢弃它，以及它左边的所有项目。我们的搜索范围缩小到只有三个项目。我们取中间的一个，发现它就是我们想要的。我们只用了三次探测就完成了搜索，甚至不需要检查 16 个项目中的 13 个。

In the beginning, the domain of our search is the full set of items. We go to the middle item, which we find out is 384. This is bigger than 135, so we discard it, along with all the items to its right. We take the middle of the remaining items, which turns out to be 72. This is smaller than 135, so we discard it, along with all the items on its left. Our search domain has shrunk to just three items. We take the middle one and find that it is the one we want. It took us only three probes to finish our search, and we did not even need to check 13 of the 16 items.

如果我们要查找不存在的条目，这个过程同样有效。您可以在下图中看到，我们在相同的条目中搜索标签为 520 的条目。

The process will also work if we are looking for something that does not exist. You can see that in the next figure, where we are searching among the same items for one labeled 520.

这次，520 大于 384，所以我们将搜索范围限制在项目的右半部分。我们发现上半部分中间的值为 613，大于 520。然后，我们将搜索范围限制为三个项目，其中中间的是 507。这小于我们的目标值 520。我们将其丢弃，现在只剩下一个项目需要检查，但我们发现它不是我们想要的。因此，我们可以结束搜索并报告搜索失败。我们只进行了四次探测。

This time, 520 is greater than 384, so we restrict our search to the right half of the items. There we find that the middle of the upper half is 613, greater than 520. Then we limit our search to just three items, the middle of which is 507. This is smaller than our target of 520. We discard it and now are left with only one item to check, which we discover is not the one we want. So we can finish our search reporting that it was unsuccessful. It took us only four probes.

我们所描述的方法被称为二分查找，因为每次我们将搜索的值域减半。我们将执行搜索的值域称为搜索空间。基于这个概念，我们可以将二分查找简化为包含以下步骤的算法：

The method we described is called binary search because each time we cut in half the domain of values in which we search. We call the domain of values where we perform our search the search space. Using this concept, we can render the binary search as an algorithm comprising these steps:

1. 1. 如果搜索空间为空，则无处可寻，报告失败并停止。否则，查找搜索空间的中间元素。
2. 1. If the search space is empty, we have nowhere to look, so report failure and stop. Otherwise, find the middle element of the search space.
3. 2. 如果中间元素小于搜索词，则从中间元素开始限制搜索空间并返回步骤 1。
4. 2. If the middle element is less than the search term, limit the search space from the middle element onward and go back to step 1.

1. 3. 否则，如果中间元素大于搜索词，则将搜索空间限制在中间元素以内，并返回步骤 1。
2. 3. Otherwise, if the middle element is greater than the search term, limit the search space up to the middle element and go back to step 1.
3. 4. 否则，中间元素等于搜索词；报告成功并停止。
4. 4. Otherwise, the middle element is equal to the search term; report success and stop.

这样，我们将要搜索的元素除以二。这是一种分而治之的方法。它会导致重复除法，正如我们在第一章中看到的，这会产生对数。重复除以二会得到以二为底的对数。在最坏的情况下，二分查找会不断地重复除法，直到无法再除为止，就像我们在失败的查找示例中看到的那样。对于n 个元素，这种情况不会发生超过次；因此，二分查找的复杂度为。

In this way, we divide by two the items that we have to search. This is a divide-and-conquer method. It results in repeated division, which as we have seen in chapter 1 gives us the logarithm. Repeated division by two gives us the logarithm base two. In the worst case, a binary search will keep dividing and dividing our items, until it cannot divide any further, like we saw in the unsuccessful search example. For n items, this cannot happen more than times; it follows that the complexity of a binary search is .

与顺序搜索，甚至是自组织搜索相比，这种改进令人印象深刻。在一百万个项目中搜索，只需 20 次探测即可。从另一个角度来看，只需 100 次探测，我们就能在 中搜索并找到任何项目，这比100 亿个项目要多得多。

The improvement compared to a sequential search, even a self-organized search, is impressive. It will not take more than 20 probes to search among a million items. Viewed from another angle, with a hundred probes we are able to search and find any item among , which is more than one nonillion.

二分查找的效率令人惊叹。它的效率或许只有它的声名远扬才能与之匹敌。它是一种直观的算法。但事实证明，这种简单的方法在计算机程序中很难正确实现。原因并非二分查找算法本身，而是我们将算法转化为实际的编程语言代码的方式，导致程序员们深受其实现中潜藏的隐蔽漏洞的困扰。而且，我们说的可不是新手；即使是世界级的程序员也未能做到完美。4

The efficiency of a binary search is astounding. Its efficiency is probably only matched by its notoriety. It is an intuitive algorithm. But this plain method has proved time and again tricky to get right in a computer program. For reasons that have nothing to do with the binary search algorithm per se, but rather the way we turn algorithms into real computer code in programming language, programmers have been prey to insidious bugs that have crept into their implementations. And we are not talking about rookies; even world-class programmers have failed to get it right.4

为了了解此类错误可能潜伏在哪里，请考虑在算法的第一步中，我们如何在要搜索的项中找到中间元素。这里有一个简单的想法：第 m个元素和第 n个元素的中间元素是，如果结果不是自然数，则四舍五入。这是正确的，并且它遵循初等数学，因此它适用于任何地方。

To get an idea of where such bugs may lurk, consider how we find the middle element among the items we want to search in the first step of the algorithm. Here is a simple idea: the middle element of the mth and nth elements is , rounded if the result is not a natural number. This is true, and it follows from elementary mathematics, so it applies everywhere.

但计算机除外。计算机的资源有限，内存就是其中之一。因此，我们不可能在计算机上表示所有我们想要的数字。有些数字太大了。如果计算机对其可处理的数字大小有上限，那么m和n都应该低于该上限。当然，低于该上限。但要计算，我们必须计算然后除以二，而这个和可能大于上限！这称为溢出：超出允许值的范围。因此，你会遇到一个你从未想过会困扰你的错误。结果将不是中间值，而是完全不同的东西。

Except in computers. Computers have limited resources, memory among them. It is not possible, therefore, to represent all the numbers we want on a computer. Some numbers will simply be too big. If the computer has an upper limit on the size of the numbers that it can handle, then both m and n should be below that limit. Of course, is below that limit. But to calculate , we have to calculate and then divide it by two, and that sum may be larger than the upper limit! This is called overflow: going beyond the range of allowable values. So you get a bug that you had never thought would bite you. The result will not be the middle value but instead something else entirely.

一旦你明白了这一点，解决方案就很简单了。你不用把中间的 计算为 ，而是。结果相同，但没有溢出。回想起来，这似乎很简单。然而，事后看来，每个人都是先知。

Once you know about it, the solution is straightforward. You do not calculate the middle as but rather . The result is the same, but no overflow occurs. In retrospect it seems simple. In hindsight, though, everybody is a prophet.

我们感兴趣的是算法，而不是编程，但作者想给那些编写或想要编写计算机程序的人一些建议。如果你发现自己苦苦思索一行代码，却发现它并没有按照你的想法运行，不要灰心丧气。如果第二天你意识到，这个 bug 确实一直就在你眼前，也不要沮丧。你怎么可能没发现呢？你并非个例。每个人都会遇到这种情况；即使是最优秀的人也会遇到。

We are interested in algorithms, not programming, here, but let the author share a bit of advice for those who write or want to write computer programs. Do not despair if you find yourself wretched poring over a line of code that does not do what you think it should do. Do not be dismayed if the following day you realize that, indeed, the bug was before your eyes all the time. How could you have failed to see it? You are not unique. It happens to all; it happens to the best.

二分查找要求元素必须经过排序。因此，为了充分利用它的优势，我们应该能够高效地对元素进行排序——这使我们能够进入下一章，学习如何使用算法对元素进行排序。

Binary search requires that the items should be sorted. So to reap its benefits, we should be able to sort items efficiently—which allows us to segue to the next chapter, where we’ll see how we can sort things with algorithms.

简单的排序方法

Simple Sorting Methods

我们的任务是对以下项目进行排序：

Our task is to sort the following items:

诚然，如果你仔细看一下这个任务，就会发现它其实很简单；这些就是从一到十的数字。但保持简单能让我们专注于排序任务的逻辑。

Admittedly, if you take a look at the task, it’s pretty trivial; these are the numbers from one to ten. But keeping things simple will allow us to concentrate on the logic of the sorting task.

首先，我们遍历所有项，找到其中最小的项。我们将它从找到的位置取出并放在第一位。最小项是 1，所以必须把它放在第一位。由于这个位置已经被占用，我们必须对当前位于第一位的 4 进行一些处理；我们不能直接把它扔掉。我们可以做的是将它与最小项交换：将最小项移到第一位，并将之前位于第一位的项移到移动最小项后空出的位置。因此，我们从这里开始，将最小项涂成黑色，

First, we go through all the items and find the minimum among them. We take it from where we found it and place it first. The minimum of the items is 1, so this must be put into the first position. As this position is already taken, we have to do something with 4, which is currently at the first position; we cannot just throw it away. What we can do is to swap it with the minimum: move the minimum item to the first position and move the item previously in the first position to the position left vacant by moving the minimum. So we go from here, where the minimum is painted black,

到这里，

to here,

其中最小值被涂成白色，以表明它处于正确、有序的位置。

where the minimum is painted white, to indicate that it is in its correct, ordered position.

现在，我们对所有数字（除了找到的最小值）执行完全相同的操作，也就是从第二个位置开始的所有数字（灰色数字）。我们找到它们的最小值，即 2，并将其与第一个未排序的数字 6 交换：

Now we do exactly the same thing with all the numbers, save for the minimum we found—that is, all the numbers from the second position onward (the gray numbers). We find their minimum, which is 2, and swap it again with the first of the unsorted numbers, 6:

再次执行相同的操作。我们处理从第三个开始的项；找到最小值 3，并将其与当前第三位的项 10 交换：

Again we do the same. We deal with the items from the third one onward; we find the minimum, which is 3, and swap it with the item currently in the third place, 10:

如果我们继续这样做，项目 4 将保持不变，因为它已经在正确的位置，并且我们将继续将项目 5 放置在其排序位置：

If we continue this way, item 4 will stay put because it is already in its correct place and we’ll go on to place 5 in its sorted position:

每次我们遍历越来越少的元素，直到找到它们的最小值。最后，我们会找到最后两个元素中的最小值，这样所有元素就排序好了。

At each point we go through fewer and fewer items to find their minimum. In the end, we’ll find the minimum of the last two items, and once we’ve done that, all our items will be sorted.

这种排序方法称为选择排序，因为每次我们都会选择未排序项中的最小值并将其放置在它应该在的位置。与我们将要研究的所有排序算法一样，选择排序对平局（即顺序相同的元素）没有问题。如果我们在检查未排序项时发现多个最小值，我们只需从中选取一个作为工作最小值。下次我们会找到平局项并将其放置在与其相等的元素旁边。

This sorting method is called selection sort because each time, we select the minimum of the unsorted items and place it where it should be. As all sorting algorithms that we will examine, selection sort has no problem with ties—that is, elements that have the same order. If we find more than one minimum when we examine the unsorted items, we just pick any one of them as our working minimum. We’ll find the tied item next time around and put it next to its equal.

选择排序是一种简单易懂的算法。它是否也是一种优秀的算法？如果我们仔细观察，就会发现，我们每次都会从待排序元素的首尾遍历，并尝试找出剩余未排序元素中的最小值。如果我们有n 个元素，那么选择排序的复杂度为。这本身并不坏；这种复杂度并不令人望而却步，我们可以在合理的时间内解决大型问题（例如对大量元素进行排序）。

Selection sort is a straightforward algorithm. Is it also a good one? If we pay attention to what we are doing, we are going from the beginning to the end of the items that we want to sort, and each time we try to find the minimum of the remaining unsorted items. If we have n items, the complexity of the selection sort is . This is not bad in itself; such complexity is not prohibitive, and we can tackle large problems (read: sort a lot of items) in a reasonable amount of time.

问题是，正因为排序如此重要，所以确实存在比它更快的算法。所以，虽然选择排序本身并不坏，但当我们手头有大量数据时，我们通常更喜欢使用其他更高级的算法。同时，选择排序并非不仅易于人类理解，而且易于在计算机上高效实现。因此，它显然不仅仅具有学术价值，而且在实践中也得到了切实的应用。

The thing is, exactly because sorting is so important, algorithms do exist that are faster than that. So although selection sort is not inherently bad, we usually prefer to use other, more advanced algorithms when we have a lot of items at hand. At the same time, selection sort is not only easy to understand by humans but is also easy to implement on a computer in an efficient way. So it is clearly not of just academic interest; it is really used in practice.

我们现在要讨论的另一种简单排序算法也同样如此。与选择排序类似，这是一种计算机之外也易于理解的排序方法。事实上，我们在纸牌游戏中可能会用这种方法来排序手牌。

The same can be said for another simple sorting algorithm that we’ll describe now. Like selection sort, this is a sorting method that is easy to understand beyond computers. In fact, this is the way we may sort our hand in a card game.

假设你在玩一种纸牌游戏，手里有十张牌（比如，你可能在玩拉米牌）。当你一张接一张地拿牌时，你想对手中的牌进行排序。我们假设牌的点数从小到大依次为：

Imagine that you play a game of cards in which you are dealt ten cards (for example, you could be playing Rummy). As you take one card after the other, you want to sort them in your hand. We assume that the card rank, from the lowest to highest, is:

事实上，在许多游戏（以及拉米牌）中，A 可以是最低和最高级别的牌，但我们假设只有一个顺序。

In fact, in many games (and Rummy), the ace can be the lowest- and highest-ranking card, but we’ll assume that there is a single order only.

每张牌都会发给你，所以你一开始手里有一张牌，接下来还有九张牌：

You are dealt each card, so you start with one card in your hand and nine cards to follow:

现在你拿到了第二张牌；它是六：

Now you get a second card; it is a six:

六和四相邻，所以你把它留在那里并拿另一张牌，结果是二：

Six is fine next to four, so you leave it there and take another card, which turns out to be two:

这次，为了保持手牌的顺序，你需要将“2”移到“4”的左边，这样“4”和“6”就向右移动一位。在拿到另一张牌“3”之前，你需要这样做：

This time, so as to keep your hand in order, you need to move two to the left of four, thus pushing four and six one position to the right. You do that before you are dealt another card, a three:

你把3放在2和4之间，然后看到下一张牌是9。这张牌已经在你手中正确的位置了。

You insert the three between the two and four, and see the next card, a nine. This is already in the right place in your hand.

您可以继续使用您的手牌 - 例如，7、Q、J、8 和 5。最后，您将得到一手排序好的手牌。

You may continue with your hand—for instance, 7, Q, J, 8, and 5. In the end, you will end up with a sorted hand.

每张新牌都会根据之前发出的牌的位置插入到正确的位置。这种排序方法被称为插入排序，它适用于任何类型的物体，而不仅仅是扑克牌。

Each new card was inserted in the right place in relation to the previous cards that had been dealt. This way of sorting is called insertion sort for that reason, and it works for any kind of objects, not just playing cards.

与选择排序类似，插入排序也易于实现。事实证明，它们的复杂度相同：。但它有一个独特的特点：就像我们的扑克牌示例一样，在对项目进行排序之前，您不需要事先知道它们。实际上，您在拿到它们时就对它们进行排序。这意味着，当要排序的项目以某种方式实时传输给您时，可以使用插入排序。我们在第 2 章讨论图表中的锦标赛问题时遇到了这种实时工作的算法，我们将其称为在线算法。如果我们必须对未知数量的项目进行排序，或者我们必须能够立即停止并在突然被要求这样做时提供排序列表，那么插入排序就是可行的方法。2

Like selection sort, insertion sort is straightforward to implement. It turns out that it has the same complexity: . It does have a distinct characteristic, though: as in our playing cards example, you don’t need to know the items in advance before you sort them. In effect, you sort them as you get them. That means that you can use insertion sort when the items to be sorted are somehow streamed to you live. We met this kind of algorithm, which works live as it were, when we discussed the tournament problem in graphs in chapter 2, and we called it an online algorithm. If we have to sort an unknown number of items, or if we must be able to stop immediately and provide a sorted list whenever we are suddenly called to do so, then insertion sort is the way to go.2

基数排序

Radix Sort

现在让我们回到霍勒瑞斯。他的制表机既没有使用选择排序，也没有使用插入排序。它们实际上使用了一种至今仍在使用的方法的前身，叫做“排序法”。基数排序。为了向第一个机器支持的排序应用程序致敬，值得花些时间了解一下基数排序的工作原理。它也很有趣，因为在这种排序方法中，待排序的项实际上并不会相互比较。至少，正如我们将看到的，并非完全如此。更重要的是，基数排序不仅具有历史意义，它的性能也非常出色。如此古老而实用的算法，还有什么理由不让人喜欢呢？3

Let us now return to Hollerith. His tabulating machines did not use selection sort, nor insertion sort. They actually used a precursor of a method still in use today, called radix sort. As a tribute to the first machine-enabled sorting application, it is worth spending some time on how radix sort works. It is also interesting because this is a sorting method in which the items to be sorted are not really compared to each other. At least not entirely, as we will see. What’s more, radix sort is not just of historical interest, as it performs fantastically. What’s not to like in a venerable yet practical algorithm?3

理解基数排序最简单的方法还是用扑克牌。假设我们有一副洗好的牌，想对它进行排序。一种方法是将其分成 13 堆，每个点值对应一堆。我们把这副牌取出来，每堆放进相应的牌堆。这样一来，我们就得到了 13 堆牌，每堆四张：一堆包含所有 A，一堆包含所有 2，等等。

The easiest way to see a radix sort is by using playing cards again. Suppose that we have a full deck of cards that has been shuffled and want to sort it. One way to do it is to form 13 piles, one for each rank value. We go through the deck, taking each card and placing it in the respective pile. We’ll get 13 piles of four cards each: a pile containing all the aces, another one containing all the twos, and so on.

然后我们把牌一叠一叠地收起来，小心地把抽到的每一叠牌放在我们收集的牌堆底部。这样，我们手中的所有牌就都整理好了，并且部分排序。前四张牌是A，接下来的四张是2，一直到K。

Then we collect the cards, pile by pile, taking care to put each pile we pick at the bottom of the cards we are collecting. In this way we’ll have all the cards in our hands, partially sorted. The first four cards will be aces, the next four twos, and all the way to the kings.

我们现在创建四个新的牌​​堆，每个花色一个。我们将检查这些牌，取出每张牌并将其放入对应的牌堆。我们将得到四堆不同花色的牌。由于牌值已经排序，所以每堆牌中都会有同一花色的所有牌，按顺序排列。

We now create four new piles, one for each suit. We’ll go through the cards, taking each card and putting it into the corresponding pile. We’ll get four piles of suits. Because the values were already sorted, in each pile we will have all cards of a single suit, in rank order.

为了完成卡片的分类，我们只需要将它们一堆一堆地收集起来。

To finish sorting our cards, we only need to collect them pile by pile.

这就是基数排序的精髓。我们并不是通过完全比较牌来对牌进行排序，而是进行部分比较，先按点数比较，然后按花色比较。

This is the essence of radix sort. We did not sort the cards by fully comparing cards between them. We performed partial comparisons, first by rank, and then by suit.

当然，如果基数排序只适用于纸牌，那么它就不值得我们关注了。我们可以看看基数排序是如何对整数进行操作的。假设我们有以下一组整数：

Of course, if radix sort was applicable only to cards, it would not merit our attention here. We can see how radix sort works with integer numbers. Suppose that we have the following group of integers:

我们确保所有整数的位数相同。因此，我们会根据需要在数字左侧填充零，例如将 5 变为 005，将 97 变为 097，将 53 变为 053。我们检查所有数字，并根据其最右边的数字进行分类。我们用这个数字将它们分成十堆：

We make sure that all the integers have the same number of digits. So we pad the numbers with zeros on the left if necessary, turning 5 to 005, 97 to 097, and 53 to 053. We go through all our numbers and triage them by their rightmost digit. We use that digit to place them in ten piles:

我们淡化了数字的填充颜色，以表明它们已部分排序；每堆数字都包含最右边数字相同的数字。第一堆数字的尾数均为零，第二堆数字的尾数均为一，直到最后一堆数字的尾数均为九。现在，我们收集这十堆数字，从左边第一堆开始，并在底部添加其他堆（注意不要以任何方式打乱数字顺序）。然后，我们根据右边第二位数字将它们重新分配到十堆中，得到：

We lightened up the numbers’ fill color to indicate that they are partially sorted; each pile contains the numbers with the same rightmost digit. All the numbers in the first pile end in zero, and in the second pile they end in one, up to the last pile, where they end in nine. We now collect the ten piles, starting from the first on the left and adding piles at the bottom (taking care not to shuffle the numbers in any way). Then we redistribute them into ten piles using the second digit from the right and get:

这次，第一堆所有数字的右起第二位都等于零；第二堆所有数字的右起第二位都等于一，其他堆也类似。同时，每堆中的物品都按其末位数字排序，因为我们第一次堆放它们时就是这么做的。

This time all the numbers in the first pile have their second from the right digit equal to zero; in the second pile they have their second from the right digit equal to one, and similarly for the other piles. At the same time, the items in each pile are sorted by their last digit because that’s what we did when we piled them the first time.

最后，我们收集这些堆并重新分配数字，这次使用右边第三位数字：

We finish by collecting the piles and redistributing the numbers, using the third digit from the right this time:

现在，每堆物品的起始数字相同，并根据第二位数字（这是上一次堆叠的结果）和最后一位数字（这是第一次堆叠的结果）进行排序。为了得到排序后的数字，我们只需最后一次收集这些物品。

Now the items in each pile start with the same digit and are sorted by their second digit, as a result of the previous piling, and their last digit, as a result of the first piling. To get our sorted numbers, we just collect the piles one final time.

基数排序可以处理单词或任何字母数字字符序列以及整数。在计算机科学中，我们将字母数字字符和符号序列称为字符串。基数排序适用于字符串；字符串可以像我们的示例中那样由数字组成，但它们可以是任何类型的字符串。字母字符串的堆数等于组成字母表的不同字符数（例如，英语有 26 个堆），但操作完全相同。基数排序的独特之处在于，即使字符串完全由数字组成，我们也将它们视为字母数字序列，而不是数字。如果您查看我们的工作方式，就会发现我们并不关心数字的值，而是每次都处理数字中的一位特定数字，就像我们从单词中提取字符一样，从右到左进行。这就是为什么基数排序有时被称为字符串排序方法。

Radix sort can work with words or any sequence of alphanumeric characters as well as integers. In computer science, we call a sequence of alphanumeric characters and symbols a string. Radix sort works with strings; the strings may be composed of digits, like in our example, but they may be any kind of strings. The number of piles for alphabetic strings will be equal to the number of distinct characters comprising the alphabet (for instance, 26 piles for English), but the operations will be exactly the same. What is distinctive in radix sort is that even when the strings are composed entirely of digits, we treat them as alphanumeric sequences, not as numbers. If you check how we worked, we did not care for the values of the numbers, but we were working each time with one particular digit from the number, in the same way that we would work by extracting characters from a word, going from the right to left. That is why radix sort is sometimes called a string sorting method.

不要被这个误导，以为基数排序可以对字符串进行排序，而我们这里介绍的其他排序方法却不行。其实它们都可以。只要组成字符串的符号本身可以排序，我们就可以对字符串进行排序。人类的名字对计算机来说是字符串，而我们可以对它们进行排序，因为字母按字母顺序排列，而名称可以按字典顺序进行比较。之所以称之为“字符串排序”，是因为基数排序将所有键（包括偶数）视为字符串。本章中的其他排序方法将数字视为数字，将字符串视为字符串，并根据情况比较数字或字符串。为了方便起见，我们在不同排序算法的示例中将数字用作键。

Do not let this fool you and lead you to think that radix sort can order strings while the other sorting methods we present here cannot. All of them can. We can sort strings, as long as the symbols that compose them can themselves be ordered. Human names are strings to computers, and we can sort them because letters are ordered alphabetically and names can be compared lexicographically. The appellation “string sorting” is because radix sort treats all keys, even numbers, as strings. The other sorting methods in this chapter treat numbers as numbers and strings as strings, and work by comparing numbers or strings, as is appropriate. It is only for convenience that we use numbers as keys in our examples in the different sorting algorithms.

基数排序通过逐位（或逐字符）处理待排序项，从而提高效率。如果我们有n 个待排序项，且每个项由w 个数字或字符组成，则该算法的复杂度为。这比选择排序和插入排序所需的复杂度要低得多。

The way radix sort works by processing the items to be sorted digit by digit (or character by character) makes it efficient. If we have n items to sort, and the items consist of w digits or characters, then the complexity of the algorithm is . That is much better than the complexity required by selection and insertion sorts.

绕了一圈，我们又回到了制表机。制表机的工作原理与此类似，它对穿孔卡片进行分类。假设我们有一副卡片，每张卡片有十列；每列上的穿孔代表一个数字。机器可以识别每列上的穿孔，从而算出相应的数字。操作员将卡片放入机器，机器根据卡片的最后一列（也就是最低有效数字）将卡片放入十个输出箱中。操作员从输出箱中收集卡片，小心不要将它们混在一起，然后将它们再次送入机器。这一次，机器根据卡片的倒数第二列（也就是最低有效数字）将卡片分配到输出箱中。最不重要的一张。重复这个过程十次后，操作员就能收集到一堆整齐的卡片了。瞧！

And so we come full circle to tabulating machines. A tabulating machine worked in a similar way, sorting punched cards. Imagine that we have a deck of cards where each card has ten columns; punched holes in each column indicate a digit. The machine could recognize the holes in each column, thus figuring out the corresponding digit. An operator put the cards in the machine, and the machine placed the cards in ten output bins depending on their last column—that is, the least significant digit. The operator collected the cards from the output bins, being careful not to mix them in any way, and fed them again into the machine, which this time distributed them into the output bins using their one but last column, the digit next to the least significant one. After repeating the process ten times, the operator could collect an ordered pile of cards. Voilà.

快速排序

Quicksort

假设有一群孩子在院子里（比如在学校）闲逛，我们想让他们按从矮到高的顺序排队。首先，我们让他们排队，他们会按照自己想要的顺序排队：

Suppose we have a group of kids milling around in a yard (perhaps at school) and want to put them in line, from the shortest to tallest. Initially we ask them to get in line, which they will do, in whatever order they want:

现在我们随机挑选一个孩子：

Now we pick a kid at random:

我们告诉孩子们四处走动，这样所有比被选中的孩子矮的孩子都应该移动到他们和所有其他人都应该向右移动。下图展示了我们选中的孩子最终的位置，你可以检查一下，个子较高的孩子在右边，个子较矮的孩子在左边：

We tell the kids to move around so that all kids who are shorter than the chosen one should move to the left of them and all the rest should move to their right. In the following figure we show where the kid we picked ended up, and you can check that those kids who are taller are to the right and those who are shorter are to the left:

我们没有要求孩子们按正确的顺序排列。我们只要求他们相对于我们选中的孩子移动。所以他们分成了两组，分别位于选中孩子的左右两侧。这两组中的孩子并没有按照从矮到高的顺序排列。然而，我们确实知道，在我们试图排列的队伍中，肯定有一个孩子排在最后：就是我们选中的那个孩子。左边所有孩子都比较矮，右边所有孩子至少都一样高。我们把选中的孩子称为“枢轴”，因为其他孩子都围绕着他移动。

We did not ask the kids to put themselves in the right order. We only asked them to move relatively to the kid we chose. So they formed two groups, on the left and right of the chosen one. The kids in these groups are not in any shorter-to-taller sequence. We do know, however, that one kid is certainly in the final position in the line we are trying to form: the very kid we picked. All the kids on the left are shorter and all the kids on the right are at least as tall. We call the kid we picked pivot because the rest of the kids have moved around them.

为了方便观察，我们将按照惯例，将摆放在正确位置的孩子涂成白色。当我们选定一个孩子作为枢轴时，我们会将其涂成黑色；当我们将其余孩子移动到枢轴周围后，我们会用一顶小黑帽来指示枢轴的最终位置（白色是因为它处于正确的位置，黑色的帽子顶部表示它是枢轴）。

As a visual aid, we will follow the convention of painting white the kids who are put in the right position. When we select a kid as a pivot, we will paint them black; when we have moved the rest of the kids around the pivot, we will use a small black hat to indicate the final position of the pivot (it’s white because it’s in the right position, with a black top, to indicate that it was the pivot).

现在我们把注意力转移到左边或右边的两组中的一组——假设是左边。我们再次从该组中随机选取一个枢轴：

Now we shift our attention to one of the two groups, left or right—say the left. Again we pick a pivot in that group at random:

我们要求该组中的孩子做与之前相同的事情：如果个子较矮，就移到枢轴的左侧，否则就移到右侧。我们将再次分成两个新的、更小的组，如下所示。其中一个组只有一个人，所以那个孩子在这个小组中的位置是正确的。然后，我们让其余的孩子在第二个枢轴的右侧。第二个枢轴在正确的位置，所有个子较矮的孩子在左侧，其余的孩子在右侧。右侧的这个组延伸到第一个枢轴。然后，我们从该组中选出一个新的第三个枢轴。

We ask the kids in that group to do the same thing as before: move so that if they are shorter, they move to the left of the pivot and otherwise they should end to the right. We will have again two new, smaller groups, which you can see below. One of them is a group of one, so that kid is in their right place in that trivial group. Then we have the rest of the kids to the right of the second pivot. The second pivot is in the right place, with all the shorter kids to the left, and all the rest to the right. This group to the right extends to the first pivot. We then pick a new, third pivot from that group.

当我们告诉小组里的孩子像之前一样，根据他们相对于第三个支点的高度移动时，他们会分成两个小组。我们把注意力集中在左边的那个小组。我们像之前一样。我们选择一个支点，也就是第四个支点，然后让这三个孩子围绕它移动。

When we tell the kids in the group to move as before, related to how tall they are with respect to the third pivot, two smaller groups will be formed. We focus our attention to the one on the left. We do as before. We pick a pivot, our fourth, and we ask the kids in this group of three to move around it.

当他们这样做时，轴心最终会是三个孩子中的第一个，所以我们剩下两个孩子在轴心的右边。我们从中选一个作为轴心，另一个孩子如果需要的话会移动到他的右边。

When they do, the pivot ends up being the first of the three, so we have a remaining group of two kids on the pivot’s right. We pick one of the pair as a pivot, and the other kid will move, if needed, to their right.

事实证明，这孩子根本不需要搬家。目前，我们已经设法把大约一半的孩子安排好了；还有两组孩子是我们之前留下的。我们正在处理之前的枢轴点。我们回到左边这两组中的第一个，以便在那里选择一个枢轴点，并重复这个过程。

It turns out that this kid does not need to move at all. Right now, we have managed to put about half the kids in order; there are two groups that we had left when we were dealing with previous pivots. We go back to the first of these two groups from the left in order to pick a pivot there and repeat the process.

同样，不需要四处走动，所以我们去找最后一组未分类的孩子来选择一个支点。

Again, no movement around was necessary and so we go to the last group of unsorted kids to pick a pivot.

我们在枢轴的右侧得到一组一元组，在枢轴的左侧得到一组二元组。我们关注左侧的组，并从中选择一个作为我们的最后一个枢轴。

We get a group of one, on the pivot’s right, and a group of two, on the pivot’s left. We focus on the left group and select one of the two as our last pivot.

好了，所有孩子都按身高排序了。

We are done. All the kids are in order of height.

让我们回顾一下我们所做的工作。我们每次都把一个孩子放到正确的位置，从而成功地把孩子们排好了序。要做到这一点，我们只需要让其他孩子围着他们移动。当然，这总是有效的，不仅适用于孩子，也适用于任何我们想要排序的东西。如果我们有一组需要排序的数字，我们可以遵循类似的过程，随机挑选一个数字，然后移动其余的数字，使较小的数字排在我们选定的数字之前，其余的则排在它之后。我们将在形成的较小的组中重复该过程；最终，我们将使所有数字都按正确的顺序排列。这就是快速排序算法的基础过程。

Let’s take stock of what we did. We managed to put the kids in order by putting one kid in their right place each time. To do that, we only needed to ask the rest of the kids to move around them. This will always work, of course, not just with kids but also with anything that we may want to sort. If we have a group of numbers that we can sort, we can follow a similar process, picking up a number at random and moving around the rest of the numbers so that those that are smaller end up before our chosen number, and the rest end up after it. We’ll repeat the process in the smaller groups that are formed; in the end, we’ll have all the numbers in the right order. This is the process that underlies the quicksort algorithm.

快速排序基于这样的观察：如果我们设法将一个元素放置在相对于所有其他元素的正确位置，无论这个位置如何，然后对剩余元素重复此操作，最终所有元素都会处于正确的位置。如果我们回想一下我们在选择排序中所做的，我们同样也取出每个元素，并使其相对于所有元素正确定位。其余元素，但我们取的元素始终是剩余元素中的最小值。这是一个关键的区别：在快速排序中，我们不应该选择剩余元素中的最小值作为基准。让我们看看如果我们这样做会发生什么。

Quicksort is based on the observation that if we manage to position one element in the correct position with respect to all the rest, whatever that position might be, and then repeat this with the remaining elements, we’ll end up with all the elements in their correct positions. If we think back on what we did with selection sort, there we also took each element and positioned it correctly with respect to all the rest, but the element we took was always the minimum of the remaining ones. This is a crucial difference: in quicksort, we should not pick the minimum of the remaining elements as our pivot. Let’s see what happens if we do so.

如果我们从同一组孩子重新开始，我们会选择所有孩子中身高最矮的那个作为轴心。他会排到队伍的最前面，其余孩子则会排到轴心后面。

If we start again with the same group of kids, we’ll get the shortest of all kids as our pivot. That one will go to the beginning of the line, and all the rest will move behind the pivot.

然后，我们选出比第一个孩子高的孩子，让他排在第二位。其余孩子也都排在中锋后面。

Then we’ll get the kid who is immediately taller than the first one and put them second in line. All the rest of the kids will go, again, behind the pivot.

对第三个孩子做同样的事情，我们得到这样的结论：

Doing the same thing with the third kid gets us to this point:

但请注意，这看起来很像选择排序，因为我们用剩下的最矮的孩子从左到右填充这条线。

But notice how this looks eerily like a selection sort, as we are filling in the line from the left to the right with the shortest of the remaining kids.

我们还没说每次如何选择一个元素作为主元。现在我们明白了，我们不应该选择元素中最小的那个。首先，选择最小值需要付出努力；我们每次都应该去寻找最小值。其次，它的行为就像我们已经知道的算法一样，所以这样做没有多大意义。

We have not said how we choose an element as a pivot each time. We now see we should not choose the minimum of the elements. First, choosing the minimum requires effort; we should really go and find the minimum each time. Second, it behaves like an algorithm we already know and so there should not be much point in doing it.

事实上，快速排序比选择排序更好，因为“通常”（我们很快就会明白“通常”的含义）我们会选择一个能够以更公平的方式划分数据的元素作为主元。选择最小元素会导致最不公平的划分：主元左侧没有元素，其余元素则全部位于主元右侧。因此，每次我们只需要设法定位主元本身即可。

The truth is that quicksort is better than selection sort because “normally” (we’ll see what normally means shortly) we’ll pick as our pivot something that partitions our data in some more equitable way. Choosing the minimum element creates the most unequal partition: nothing on the left of the pivot, and all the rest to the right of the pivot. Each time, then, we just manage to position the pivot itself.

如果划分得更好，那么我们不仅能定位主元，还能将主元左侧的所有元素相对于主元右侧的元素放置在正确的位置。是的，它们还没有到达最终位置。但总体来说，它们的位置比之前更好了。所以我们有一个元素，也就是枢轴，处于最佳位置，其他元素的位置也比之前更好。

If the partition is better, then we do not just manage to position the pivot. We also manage to position all the elements to the left of the pivot in their correct positions with respect to the elements to the right of the pivot. Yes, they are not in their final positions yet. But overall, they are in better positions than before. So we have one element, the pivot, in the best position possible, and the other elements better positioned than before.

这对快速排序的性能有重要影响：它的预期复杂度为，远优于。如果我们要对 100 万个项目进行排序，则需要，即 1 万亿，而约为 2000 万。

This has an important effect on the performance of quicksort: its expected complexity is , which is way better than . If we want to sort 1 million items, works out to , a trillion, while is about 20 million.

一切都取决于选择合适的数据透视表。每次都寻找一个能够以最佳方式划分数据的透视表毫无意义；这需要反复搜索才能找到正确的透视表，这会增加流程的复杂性。因此，一个好的策略是听天由命。只需随机选择一个透视表，然后使用它来划分数据即可。

It all hinges on picking the proper pivot. Searching for a pivot that would partition our data in the best possible way each time does not make sense; it would require searching to find the right pivot, so this would add complexity to the process. A good strategy, then, is to leave it to chance. Just pick a pivot at random and use what you picked to partition the data.

要理解为什么这是一个好的策略，让我们先看看为什么它不是一个坏策略。如果它导致像我们刚才看到的那样的行为，即快速排序退化为选择排序，那么它就是一个坏策略。如果我们每次选择一个实际上没有划分元素的项作为枢轴，就会发生这种情况。如果我们每次都选择项中的最小值或最大值（情况完全相同），也会出现这种情况。所有这些情况发生的总体概率可以得出为

To see why this is a good strategy, let us see why it is not a bad one. It would be a bad one if it led to a behavior like the one we just saw, where quicksort degenerates to selection sort. This would happen if we pick each time as a pivot an item that does not really partition the elements. This can happen if we pick each time the minimum or maximum of the items (the situation is exactly the same). The overall probability of all this happening can be found to be

这种概率很难把握，因为它极低。具体来说，如果你拿一副牌52张扑克牌，随机洗牌，最终牌堆顺序正确的概率是这大约相当于抛硬币连续226次正面朝上的概率。乘以后，情况并没有太大改善。这个数字大约等于。从宇宙的角度来看，地球由大约个原子组成。如果你和你的朋友分别从地球上挑选一个原子，你们挑选出相同原子的概率是，实际上大于——对一副牌进行病态快速排序的概率。4

A probability such as is hard to grasp because it is abysmally low. To put it into context, if you take a deck of 52 playing cards and shuffle it randomly, the probability that the deck will end up being in order is This is about the same as flipping a coin and coming out heads 226 times in a row. When you multiply by , things are not improved much. The number is approximately equal to . To put the matter in cosmic perspective, the earth is composed of about atoms. If you and a friend of yours were to pick independently an atom from the earth, the probability that you would pick the same atom would be , actually greater than —the probability of pathological quicksort on a deck of cards.4

这就解释了为什么我们“通常”会以更公平的方式选择枢轴，正如我们上面所说。除非运气特别差，否则我们不会每次都选择最糟糕的枢轴。实际上，概率对我们更有利：通过随机选择枢轴，我们预期复杂度为。理论上也有可能比这更糟，但这种可能性仅仅出于学术研究。快速排序的速度与我们预期的一样快，适用于所有实际用途。

That explains that “normally” we pick a pivot in a more equitable way, as we said above. Excepting a streak of bad luck of cosmic proportions, we do not expect to pick the worst pivot possible each time. The odds actually work better in our favor: it is by picking pivots at random that we expect to get a complexity of . It is theoretically possible to do worse than that, but the possibility is only of academic interest. Quicksort will be as fast as we expect it to be for all practical purposes.

快速排序由英国计算机科学家托尼·霍尔于 1959-1960 年间开发。5它可能是当今最流行的排序算法，因为如果正确实现，它的性能会优于所有其他算法。它也是我们所见过的第一个行为并非完全确定的算法。虽然它总是能正确排序，但我们无法保证它始终具有相同的运行时性能。但我们可以保证它极不可能表现出病态行为。这是一个重要的概念，它将我们引向了所谓的随机算法：那些在操作中使用偶然因素的算法。这与我们的直觉相反；我们期望算法是终极确定性野兽，沿着预定的路径盲目地遵循我们为它们设定的指令。然而，近年来随机算法蓬勃发展，因为事实证明，偶然性可以帮助我们解决那些仍然难以用更标准的方法解决的问题。6

Quicksort was developed by the British computer scientist Tony Hoare in 1959–1960.5 It is probably the most popular sorting algorithm today because when implemented correctly, it outruns all others. It is also the first algorithm that we see whose behavior is not entirely deterministic. Although it will always sort correctly, we cannot guarantee that it will always have the same runtime performance. We can guarantee that it is extremely unlikely that it will exhibit pathological behavior. This is an important concept, which brings us to the so-called randomized algorithms: those algorithms that use an element of chance in their operation. This runs contrary to our intuition; we expect algorithms to be the ultimate deterministic beasts, slavishly following the instructions we lay down for them on a preordained path. And yet randomized algorithms have blossomed in recent years, as it has turned out that chance can help us solve problems that remain intractable to more standard approaches.6

合并排序

Merge Sort

我们已经了解了基数排序，它本质上是按照分布对数据进行排序：在每一轮数据处理中，它将每个数据放入正确的堆中。现在我们将了解另一种排序方法，它通过合并数据而不是拆分数据来对数据进行排序。这种方法称为归并排序。

We’ve met radix sort, which essentially sorts items by distribution: in each round through the data, it places each item in a correct pile. Now we’ll meet another sorting method, which sorts item by merging stuff together instead of splitting them apart. The method is called merge sort.

归并排序首先承认排序能力有限；想象一下，如果我们把元素以任何随机的排列方式交给我们，我们无法对它们进行排序。我们只能做到以下几点：如果我们有两组项目，并且每个组已经排序，我们可以将它们合并在一起并得到一个排序的组。

Merge sort starts by admitting to a limited capability for sorting; imagine that we are unable to sort our items if they are given to us in any random arrangement. We are only able to do the following: if we are given two groups of items, and each group is already sorted, we can merge them together and get a single, sorted group.

例如，假设我们有以下两个组，每行一个（尽管在我们的示例中，这两个组具有相同数量的项目，但组的大小不需要相等）：

For example, say we have the following two groups, one per row (although in our example the two groups have the same number of items, there is no need for the groups to be equal in size):

如你所见，这两个组都已经排序了。我们想将它们合并，创建一个单独的排序组。这很简单。我们检查两个组的第一个元素。我们发现 15 小于 21，所以这将是我们第三个组的第一个元素：

As you can see, each of the two groups is already sorted. We want to merge them in order to create a single sorted group. This is really simple. We check the first item of both groups. We see that 15 is smaller than 21, so this will be the first item of our third group:

我们再次检查这两个组的第一个元素，这次第二个组中的 21 小于第一个组中的 27。因此，我们将其取出并附加到第三个组中。

We examine again the first elements of the two groups, and this time 21 from the second group is smaller than 27 from the first group. So we take it and append it to the third group.

如果我们继续这样做，我们将从第一组中取出 27 个，然后从第二组中取出 35 个，将它们添加到第三组的末尾：

If we continue in this way, we’ll take 27 from the first group and then 35 from the second group, adding them to the end of the third group:

现在 51 小于 59，56 也小于 59。由于我们已经将 35 从第二组移到了第三组，最终我们将连续移动三个元素从第二组移到了第三组。这没问题，因为这样我们就能保持第三组中元素的排序。前两组没必要以相同的速率减少。

Now 51 is smaller than 59, and 56 is smaller than 59. As we already have moved 35 from the second group to the third, in the end we’ll have moved three items in a row from the second group to the third. That is fine because in this way we keep items in the third group sorted. There is no reason why the two first groups should diminish at the same rate.

我们回到第一组，因为 59 小于 69，所以我们将其添加到第三组：

We return to the first group, as 59 is smaller than 69, so we add it to the third group:

接下来，通过将 69 移到第三组，我们完全清空了第二组：

Next, by moving 69 to the third group we empty the second group completely:

最后，我们将第一组中剩余的元素移动到第三组中——它们肯定比第三组中的最后一个元素大，否则我们之前就不会把它移到那里了。现在我们的元素已经完全排序了：

We finish by moving the last remaining elements of the first group to the third group—they are definitely larger than the last element of the third group or otherwise we would not have moved it there previously. Our items are completely sorted now:

能从两个已排序的组生成一个已排序的组固然不错，但这似乎并不能解决我们对一组未排序元素进行排序的问题。确实如此，但它仍然是解决方案的重要组成部分。

It’s nice to have a way of producing a sorted group from two sorted groups, but this does not seem to solve our problem of sorting a single group of unsorted items. It is true it does not, yet it is an important component of the solution.

假设我们有一群人。我们把一组物品交给其中一人分类。这个人不知道如何分类，但他知道，如果他们能找到两份已排序的物品，就能得到最终排序好的一组。所以他们的做法是：把这组物品分成两份，然后交给另外两个人。他们对第一个人说：“拿着这组物品去分类。分类完成后，归还给我。” 他们对第二个人说了同样的话。然后他们等待。

Imagine now that we have a group of people. We give to one of them a group of items to sort. That person does not know how to sort them, but they do know that if somehow they had two sorted parts of the items, they could produce a final sorted group. So what they do is this: they split the group in two and pass it on to two other people. They say to the first of them, “Take this group and sort it. Once you are done, return it to me.” They say the same thing to the second person. Then they wait.

虽然我们的第一个联系人不知道如何对物品进行排序，但如果两个新联系人设法对自己的物品进行排序并归还，那么第一个联系人就会将最终排序好的物品归还给我们。但是，另外两个联系人的知识并不比我们的第一个联系人多——他们不知道如何排序，而只知道如何使用上述算法合并排序好的物品——那么，这真的取得了什么成果吗？

Although our first point of contact does not know how to sort the items, if the two new contacts manage somehow to sort their own parts and return them, then the first person would return to us the final, completely sorted group. But the two other contacts know no more than our initial contact—they don’t know how to sort but rather only how to merge sorted stuff using the algorithm above—so has anything really been achieved?

答案是肯定的，只要他们做同样的事情：他们将自己的部分分成两部分，每个人将自己的部分委托给另外两个人，等待他们执行命令并为他们提供两个排序好的部分。

The answer is yes, provided that they do the same: they split their part in two, and each delegates their part to two other persons, waiting for them to do their bidding and provide them with two sorted parts.

这看起来像是一个终极的推卸责任的游戏，但如果我们试着用一个例子来说明会发生什么。我们从数字 95、59、15、27、82、56、35、51、21 和 79 开始。我们把它们交给 Alice (A)，她把它们分成两份，然后传给 Bob (B) 和 Carol (C)。你可以在下面的倒置树的第一层看到这一点：

This seems like the ultimate pass-the-buck game, but look at what happens if we try to see it unfold with an example. We start with the numbers 95, 59, 15, 27, 82, 56, 35, 51, 21, and 79. We give them to Alice (A), who splits them in two, and passes them to Bob (B) and Carol (C). You can see that in the first level of the upside-down tree below:

然后，鲍勃将他的号码一分为二，分别传给戴夫（D）和伊芙（E）。同样，卡罗尔也一分为二，分别传给弗兰克（F）和格蕾丝（G）。我们这群人继续互相推诿责任。戴夫将他的号码分给了海蒂（H）和伊万（I）；伊芙将她的两个号码分给了朱迪（J）和凯伦（K）；弗兰克和格蕾丝分别分给了利奥（L）和马洛里（M）以及尼克（N）和奥利维亚（O）。最后，海蒂将她的一对号码分给了佩吉（P）。和昆汀 (Q)，而里奥则将他的搭档分给了罗伯特 (R) 和西比尔 (S)。

Then Bob splits his numbers into two, and passes them on to Dave (D) and Eve (E). Similarly, Carol splits her numbers, and passes them on to Frank (F) and Grace (G). Our cast of characters continue passing the buck. Dave divides his numbers to Heidi (H) and Ivan (I); Eve distributes her two numbers to Judy (J) and Karen (K); Frank and Grace split to Leo (L) and Mallory (M) and Nick (N) and Olivia (O), respectively. Finally, Heidi splits her pair to Peggy (P) and Quentin (Q), while Leo splits his pair to Robert (R) and Sybil (S).

树叶子旁的人其实没什么事可做。佩吉和昆汀每人收到一个数字，并被告知要对它进行排序。但根据定义，单个数字是有序的：它本身就有序。所以佩吉和昆汀只需把他们的数字还给海蒂即可。此外，伊万、朱迪、凯伦、罗伯特、西比尔、马洛里、尼克和奥利维亚也归还了他们收到的数字。

The people at the leaves of the tree have really nothing to do. Peggy and Quentin receive a number each, and they are told to sort it. But a single number is sorted by definition: it is in order with itself. So Peggy and Quentin just give their number back to Heidi. Also, Ivan, Judy, Karen, Robert, Sybil, Mallory, Nick, and Olivia return the numbers they received.

现在让我们来看下一页的树。在这棵树中，我们将从顶部的叶子（所以它看起来像一棵正常的树，而不是倒置的）移动到底部的根。让我们专注于海蒂。她得到了两个数字，每个数字都（显然）是排序的。海蒂知道如何合并两个已排序的组来得到一个组，这样她就可以用 95 和 59 得到 59, 95。然后，她将这个已排序的两组数字还给戴夫。利奥也会采取同样的行动：他会得到 35 和 56，这两个数字本身已经排序好了，并且知道如何将这两个数字按顺序排列，得到 35, 56，然后把这两个数字还给弗兰克。

Now let’s move to the tree on the next page. In this tree we’ll move from the leaves, at the top (so this looks like a normal tree, not upside down), to the root at the bottom. Let’s concentrate on Heidi. She gets back two numbers, each one of which is (trivially) sorted. Heidi knows how to merge two sorted groups to produce a single group so she can use 95 and 59 to make 59, 95. She then returns this sorted group of two to Dave. Leo will act the same: he will get 35 and 56, which are already sorted (by themselves), and knows how to put these two in order and create 35, 56, which he returns to Frank.

戴夫对最初收到的数字 95、59、15 一无所知，现在从海蒂那里得到了 59、95，从伊万那里得到了 15。这两组数字都已经排序，这意味着戴夫可以将它们合并，得到 15、59、95。同样，弗兰克从利奥那里得到了 35、56，从马洛里那里得到了 51，可以得到 35、51、56。

Dave, who was clueless about the numbers 95, 59, 15 that he had initially received, now gets 59, 95 from Heidi and 15 from Ivan. Both of these groups are already sorted, which means that Dave can merge them to create 15, 59, 95. In the same way, Frank gets 35, 56 from Leo and 51 from Malory, and can produce 35, 51, 56.

如果每个人都采取相同的方式，当数字到达 Alice 手中时，她会收到两个排序好的列表，一个来自 Carol，一个来自 Bob。她会将这两个列表合并起来，创建最终的排序列表。

If everybody acts in the same way, when the numbers reach Alice, she will get two sorted lists, one from Carol and one from Bob. She will merge them to create the final sorted list.

这两棵树是归并排序的精髓。我们尽可能地委托排序，直到无法进行任何排序，因为单个元素已经按照定义排序。然后，我们合并越来越大的组，直到将所有元素合并到一个最终排序好的组中。

These two trees are the essence behind merge sort. We delegate the sorting as much as we can, to the point that no sorting can take place because lone items are already sorted by definition. Then we merge larger and larger groups, until we absorb all elements in a single, final, sorted group.

我们对角色的智能要求极低。在第一棵树中，你可以看到Eve从Bob那里得到了一组数字，它们碰巧已经排序了：27，82。这无关紧要。她没有停下来检查它们是否需要排序——我们也不希望她这样做，因为这样的检查很耗时。她只是把它们拆分并传递下去。她会把它们取回来并合并。让他们来制作她已经得到的东西。没关系；从总体上看，Eve、Judy 和 Karen 之间这段毫无意义的三人舞不会影响算法的性能。

The smarts that we require from our characters is minimal. You can see in the first tree that Eve got from Bob a group of numbers that as it happened was already sorted: 27, 82. It does not matter. She does not stop to check whether they need sorting or not—and we don’t want her to because such a check would take time. She just splits and passes them down. She will get them back and merge them to produce what she already got. That’s all right; in the large scheme of things, this gratuitous pas de trois between Eve, Judy, and Karen won’t affect the performance of the algorithm.

归并排序的复杂度与快速排序相当。这意味着我们有两种性能相同的算法。在实践中，程序员可能会根据其他因素选择其中一种。通常，快速排序程序比归并排序程序运行得更快，因为它们在编程语言中的具体实现速度更快。归并排序先拆分数据，然后再合并它们，这意味着它们可以并行化，这样就可以利用计算机集群对大量数据进行排序，其中每台计算机都充当我们上面提到的人工排序员。

The complexity of merge sort is as good as that of quicksort, . That means that we have two algorithms with the same performance. In practice, programmers may choose one or the other depending on additional factors. Usually quicksort programs run faster than merge sort ones because their concrete implementation in a programming language is faster. Merge sort splits the data before merging them, which means that they can be parallelized, so that vast amounts of data can be sorted by a computer cluster, where each computer acts like our human sorters above.

归并排序与计算机一样古老。它的发明者是一位匈牙利裔美国人，诺伊曼·亚诺什·拉约什（Neumann János Lajos），他的美名约翰·冯·诺伊曼（John von Neumann，1903-1957）更为人熟知。1945年，他用墨水写了一份长达23页的手稿，内容是最早的数字计算机之一——电子离散变量自动计算机（EDVAC）。在第一页的顶部，他用铅笔写下了“绝密”（TOP SECRET）一词（后来被擦掉），因为计算机相关工作在1945年因与军方的联系而被列为机密。这篇论文的主题是计算机的一个非数值应用：排序。冯·诺伊曼描述的方法就是我们现在所说的归并排序。7

Merge sort is as old as computers. Its inventor was a Hungarian American, Neumann János Lajos, better known under his American name, John von Neumann (1903–1957). In 1945, he wrote a manuscript, in ink, 23 pages long, for one of the first digital computers, the Electronic Discrete Variable Automatic Computer, or EDVAC for short. At the top of the first page, the phrase “TOP SECRET” was penciled in (and later erased), as work on computers was classified in 1945 due to its connections with the military. The subject of the paper was a nonnumerical application of computers: sorting. The method that von Neumann described was what we now call merge sort.7

基本原则

The Basic Principles

当我们浏览某个网页时，该网页上的链接会指向与我们当前浏览的网页相关的其他网页。链接的存在本身就表明其末尾的网页很重要——否则网页的作者一开始就不会链接到它。请考虑下面的示例图，它表示一小部分相互链接的网页：

When we are on a web page, the links on that page point to other pages that are relevant to the page we are currently browsing. The very existence of the link indicates that the web page at the end of the link is important—otherwise the author of the web page would not link to it in the first place. Consider the example graph below, representing a small set of web pages that link to each other:

在这样的图中，我们将指向网页的链接称为反向链接；延伸开来，我们也将指向网页的页面称为反向链接。这样，网页 3 的反向链接就是指向它的边、它的入边，以及它们发出的节点：网页 2、4 和 5。在本章中，我们将关注由网页组成的图表，我们将互换使用术语“节点”和“页面”。

In such a graph, we call the links that point to a web page backlinks; by extension, we will also call the pages that point to a web page backlinks. In this way, the backlinks of web page 3 are the edges pointing to it, its incoming edges, as well as the nodes from which they emanate: web pages 2, 4, and 5. As in this chapter we will be concerned with graphs that are made up of web pages, we will be using the terms “node” and “page” interchangeably.

我们将基于两个基本原则构建一个用于查找每个网页重要性的算法：

We will build an algorithm for finding the importance of each web page based on two basic principles:

1. 1. 一个网页的重要性取决于链接到它的网页的重要性，也就是其反向链接的重要性。
2. 1. The importance of a web page depends on the significance of the web pages that link to it—that is, on the importance of its backlinks.
3. 2. 一个网页将其重要性平均分配给它所链接的网页。
4. 2. A web page divides its importance evenly over the web pages to which it links.

假设我们想计算第 3 页的重要性。我们发现它的反向链接分别为 2、4 和 5。我们依次计算每个反向链接，并假设我们知道它们各自的重要性。第 2 页将其重要性分配给第 3 页和第 5 页，因此第 3 页的重要性占其重要性的一半。第 4 页也将其重要性分配给第 3 页和第 1 页，因此第 3 页的重要性占其重要性的一半。最后，第 5 页将其重要性分配给第 2、3 和 4 页，因此第 3 页的重要性占其重要性的三分之一。为了节省输入，我们用第i页的重要性表示；r代表排名。那么第 3 页的重要性将是：

Say we want to find the importance of page 3. We saw that its backlinks are 2, 4, and 5. We take each one of them in turn and assume we know their own significance. Page 2 divides its importance over pages 3 and 5, and therefore will give half its importance to page 3. Page 4 also divides its importance over two pages, 3 and 1, and hence will give half its significance to page 3. Finally, page 5 divides its importance over pages 2, 3, and 4, and thus will give a third of its importance to page 3. To save typing, let us denote by the importance of page i; r will stand for rank. Then the importance of page 3 will be:

一般来说，如果我们想找出某个网页的重要性，并且我们知道每个反向链接的重要性，那么很容易找到我们想要的东西：将每个反向链接页面的重要性除以其链接到的网页数量，然后将结果添加到该页面的其他反向链接的贡献中。

In general, if we want to find out the importance of a certain web page and we know the importance of each backlink, it is easy to find what we are looking for: divide the importance of each backlink page by the number of web pages it links to and add the result to the contributions of the other backlinks of the page.

您可以将重要性的计算视为网页之间的投票竞赛。每个投票页面都具有一定的重要性，可以将其作为对其认为重要的网页的投票。如果它只认为一个网页重要，它就将票投给该网页。但是，如果它认为多个网页重要，那么它会将投票分成两部分，分别投给这些网页。因此，如果一个网页想将三个网页评为重要，它会将三分之一的选票投给每个网页。网页会将选票分配给哪些网页？分配给其超链接末尾的网页，也就是它链接到的网页。那么，网页的重要性是如何得出的呢？取决于其反向链接的重要性。

You may think of the calculation of importance as a voting contest between web pages. Each voting page has some significance, which it can use as a vote for those web pages that it deems important. If it considers only one web page as important, it just gives its vote to that web page. But if it considers more than one web page as significant, then it splits its vote and gives a part of the vote to each of these web pages. Therefore, if a web page wants to vote three web pages as being important, it will give to each one of them one-third of its vote. To which pages will a web page apportion its vote? To those at the end of its hyperlinks—that is, to those to which it links. And how is the importance of a web page derived? From the importance of its backlinks.

这两个原则确实赋予了网页排名某种民主的氛围。没有一个权威机构能够决定什么是最重要的。如果其他网页认为某个网页重要，并用其链接投票，那么该网页就很重要。然而，与大多数现实世界选举中适用的一人一票原则不同，并非所有网页都拥有平等的投票权。一个网页的投票页面是否可用取决于它的重要性——同样，这又是由其他网页决定的。

The two principles do endow some aura of democracy to the ranking of web pages. There is no single authority that decides what is most significant. A web page is important if other web pages think it is important, and they vote with their links. In contrast with the one person, one vote principle that holds in most real-world elections, however, not all web pages have equal votes here. The votes of a web page depend on how important it is—which, again, is determined by the other web pages.

这看起来像是诡辩，因为它实际上告诉我们，要计算一个网页的重要性，我们必须计算其反向链接的重要性。如果按照同样的推理，要计算每个反向链接的重要性，我们必须计算该反向链接的反向链接的重要性。然后，这个过程似乎越来越倒退，从一个反向链接到另一个反向链接，最后，我们不知道如何从头开始计算网页的重要性。更糟糕的是，我们可能会发现自己在兜圈子。在我们的例子中，要计算第 3 页的重要性，我们需要第 2、4 和 5 页的重要性。要计算第 2 页的重要性，我们需要第 1 页（以及第 5 页，但我们先不谈这一点）的重要性。要计算第 1 页的重要性，我们需要第 4 页的重要性，而要计算第 4 页的重要性，我们需要知道第 3 页的重要性。我们又回到了原点。

This may seem like casuistry because in effect it tells us that to find the importance of a web page, we must find the importance of its backlinks. If we follow the same reasoning, to find the importance of each of its backlinks, we must find the importance of that backlink’s backlinks. Then the process seems to regress more and more, from backlinks to backlinks, and in the end, we are left without knowing how to calculate the significance of the web page from where we started. Worse, we may find out that we run in circles. In our example, to calculate the importance of page 3, we need the importance of each of pages 2, 4, and 5. To calculate the importance of page 2, we need the importance of page 1 (and page 5, but let us leave that aside for a bit). To calculate the importance of page 1, we need the importance of page 4, and to find that, we need to know the importance of page 3. We are back where we started.

一个例子

An Example

为了解决这个问题，我们假设在计算网页的重要性之前，我们赋予所有网页同等的重要性。用投票的比喻来说，我们赋予每个网页恰好一票。投票开始时，每个页面都会按照我们描述的方式投票，将其投票分散到其链接的页面。然后，每个页面都会收到来自其所有反向链接的投票。投票转移如下：

To see how we get out of the problem, let us assume that before we begin calculating the importance of the web pages, we give them all equal significance. In terms of our voting metaphor, we give each web page exactly one vote. When the voting starts, each one of the pages will vote in the way we described, spreading its vote to the pages to which it links. Each page will then receive votes from all its backlinks. The transfer of votes will look like this:

页面 1 将其投票发送给它唯一链接的页面 2。页面 2 将其投票分成两部分，分别发送给页面 3 和页面 5。页面 3 将其投票分成三部分，分别发送给页面 1、页面 4 和页面 5。页面 4 和页面 5 使用相同的方法进行投票。

Page 1 sends its vote to page 2, the only page it links to. Page 2 divides its vote into two parts, and sends to page 3 and to page 5. Page 3 divides its vote into three parts and sends to each of pages 1, 4, and 5. Pages 4 and 5 vote using the same method.

投票结束后，每个页面将根据其反向链接获得的投票总和（或投票分数）计算总分。例如，页面 1 获得了来自页面 3 和页面 4 的投票，因此将获得 1/2 + 1/3 的投票。= 5 / 6 票，而页面 3 已获得来自页面 2、4 和 5 的投票，因此将获得投票。我们看到，页面 1 的投票份额与开始时相比有所减少，而页面 3 的投票份额有所增加。

Once voting is over, each page will calculate the total from the sum of the votes, or fractions of the votes, it has received from its backlinks. For example, page 1, having received votes from pages 3 and 4, will have 1 / 2 + 1 / 3 = 5 / 6 votes, while page 3, having received votes from pages 2, 4, and 5, will have votes. We see that page 1 decreased its share of votes compared to where it started, while page 3 increased it.

现在让我们稍微改变一下设置。我们不再在投票开始前为每个页面投一票，而是为每个页面投一票，这样所有投票加起来就是一票。一般来说，如果我们有n 个页面，我们会给每个页面投一票。其余过程完全相同。所有网页的总体重要性都等于一，并且重要性再次均匀分布在所有网页上。

Now let us change the setup a little bit. Instead of giving each page one vote before the voting starts, we give each page of a vote so that all votes sum up to one. In general, if we have n pages, we give votes to each one of them. The rest of the process is exactly the same. The overall importance of all web pages is equal to one, and the importance is again distributed evenly over all the web pages.

投票结束后，每个网页的重要性都会发生变化。如果我们进行计算，就会发现它们的重要性并非全部等于，而是依次为 0.17、0.27、0.27、0.13 和 0.17。网页 2 和 3 的重要性有所提升，而网页 1、4 和 5 的重要性则有所下降。所有网页的重要性总和为 1。

After the voting ends, the importance of each web page will have changed. Instead of having all of them equal to , if we do the calculations, we will find that they will be equal to 0.17, 0.27, 0.27, 0.13, and 0.17 for each of the pages in turn. Web pages 2 and 3 have gained in importance, while web pages 1, 4, and 5 have lost importance. The total significance of all web pages sums up to one.

现在，我们可以开始新一轮投票，规则完全相同。页面会将获得的投票分配给它们所链接的页面。第二轮投票结束时，每个页面将计算其投票数，以确定其累计重要性排名。计算后，新的重要性值将分别为 0.16、0.22、0.26、0.14 和 0.22。

We can now start another voting round, with exactly the same rules. The pages will spread the votes they have gathered to the pages to which they link. At the end of this second round, each page will count its votes to determine its standing in terms of accumulated importance. After the calculations, the new importance values will be 0.16, 0.22, 0.26, 0.14, and 0.22.

我们将再次执行完全相同的流程。实际上，我们会一遍又一遍地重复投票。如果这样做，投票结果（即分配给每个页面的重要性）将如下表所示变化，该表显示了每轮投票后的初始值和结果：

We’ll do exactly the same process again. In fact, we’ll repeat the voting again and again. If we do that, the votes—that is, the importance apportioned to each page—will evolve as in the following table, which shows the initial values and results after each voting round:

如果我们继续进行第七轮投票，我们会发现情况与第六轮投票相比保持不变。投票数以及网页的重要性将保持不变。这就得到了最终结果。网页的排名是：第 3 页最重要，其次是第 2 页，然后是第 5 页，然后是第 1 页，最后是第 4 页。

If we go on to perform another, seventh voting round, we’ll discover that the situation will remain unchanged with respect to the sixth voting round. The votes, and therefore the importance of the web pages, will remain the same. This then gives us our final result. The ranking of the web pages is that page 3 is the most important, followed by page 2, then page 5, then page 1, and last comes page 4.

让我们回顾一下我们做了什么。我们首先从两个原则入手，它们为我们提供了计算网页重要性的规则，前提是我们知道每个反向链接的重要性。在开始之前，我们设置了所有n 个重要性相等的网页，每个网页的反向链接分享量都等于 。然后，我们通过计算每个网页的重要性值来计算其重要性。这将为每个网页赋予新的重要性值，这些值与我们开始时的值不同。我们从这些值开始重复该过程。我们找到了另一组值。经过多次重复后，我们发现情况趋于稳定：重要性度量在每次重复之间不会发生变化。此时，我们停止并报告我们找到的值。

Let’s step back and reflect on what we did. We started with two principles that give us rules for calculating the importance of a web page, provided we know the importance of each of its backlinks. Before we start, we set up all n web pages with equal importance, equal to . Then we calculate the significance of each web page by summing the shares it gets from its backlinks. This gives us new values for the significance of each web page, different from the value from where we started. We repeat the process beginning with these values. We find another set of values. After a number of repetitions of this process, we found that the situation stabilized: the measure of importance would not change from one repetition to the next. At this point we called it a stop and reported the values that we found.

当然，问题在于我们刚才描述的方法是否适用于一般情况，而不是我们选择的特定示例。此外，它是否能产生合理的结果？

The question of course is whether the approach that we have just described works in general and not in the particular example that we chose. Moreover, does it produce sensible results?

超链接矩阵和功率法

The Hyperlink Matrix and Power Method

通过页面反向链接的重要性来计算页面重要性的方法有一个简洁的公式。我们从描述网页之间链接的图表开始。我们可以用一个数字矩阵来表示一个图表，我们称之为邻接矩阵。构造方法很简单。我们创建一个矩阵，其行数和列数与图中的节点数相同。然后，​​我们为每个与链接对应的交点放置一个其余所有交点均为零。本例中的邻接矩阵为：

The method of calculating the importance of a page from the importance of its backlinks has an elegant formulation. We start from the graph that describes the links between our web pages. We can represent a graph by using a matrix of numbers, which we call its adjacency matrix. The construction is straightforward. We create a matrix with as many rows and columns as the nodes in the graph. Then we put one for each intersection that corresponds to a link and zero for all other intersections. The adjacency matrix for our example is:

我们还可以使用单行或向量来表示网页的重要性：

We can also represent the importance of the web pages using a single row or vector:

现在我们深入探讨 PageRank 算法的具体细节，开始使用“PageRank”一词来表示网页的重要性。您会发现，使用这个术语是合理的，因为我们能够根据重要性得出网络上所有网页的排名。由于我们的行包含所有 PageRank，我们将其称为图表的PageRank 向量。

As we now get into the nuts and bolts of the PageRank algorithm, we’ll start using the term pagerank to refer to the significance of a web page. You will see that the term will be justified as we will be able to derive a ranking, in terms of importance, of all the pages on the web. As our row contains all the pageranks, we will call it the pagerank vector of our graph.

每个网页的重要性被分摊到它所链接的页面上。现在我们有了邻接矩阵，我们可以这样实现：对每一行进行操作，用该行中每个“1”除以该行中“1”的数量。这相当于用每个页面的投票数除以指向该页面的出站链接数量。这样做，我们得到以下矩阵：

The importance of each web page is divided over the pages to which it links. Now that we have the adjacency matrix at hand, we can do that by going to each row and dividing each one in the row by the number of ones in that row. This is equivalent to dividing each page’s vote by the number of outgoing links to that page. If we do that, we get the following matrix:

我们将这个矩阵称为超链接矩阵。

We call this matrix the hyperlink matrix.

如果我们仔细观察超链接矩阵，会发现每一列都显示了页面的重要性是如何从链接到它的页面中推导出来的。以第一列为例，它与页面 1 的重要性相关。该页面的重要性源于页面 3 和页面 4。页面 3 将其重要性赋予了页面 1，因为它链接到了三个页面；页面 4 将其重要性赋予了页面 1，因为它链接到了两个页面。图表中其他页面对页面 1 的重要性为零，因为它们没有链接到它。我们可以将其表示为：

If we look carefully at the hyperlink matrix, each column shows how the importance of a page is derived from the pages that link to it. Take the first column, which relates to the importance of page 1. This page takes its significance from pages 3 and 4. Page 3 gives of its importance to page 1 because it links to three pages, and page 4 gives of its importance to page 1 because it links to two pages. Page 1 receives zero significance from the other pages in the graph because they do not link to it. We can express this as:

但这正是 的定义，即第 1 页的 PageRank。我们通过将 PageRank 向量的元素与超链接矩阵第一列的相应元素的乘积相加来获得 PageRank。

But this is exactly the definition of , the pagerank of page 1. We got the pagerank by summing the products of the elements of the pagerank vector with the corresponding elements of the first column of the hyperlink matrix.

让我们看看如果我们取 pagerank 向量并将其元素的乘积与超链接矩阵第二列的相应元素相加会发生什么：

Let’s see what is happening if we take the pagerank vector and sum the products of its elements with the corresponding elements of the second column of the hyperlink matrix:

这正是 （第 2 页的 PageRank） 的定义。PageRank 向量元素与超链接矩阵第三列内容的乘积之和同样会给我们（第 3 页的 PageRank）：

That is exactly the definition of , the pagerank of page 2. The sum of the products of the elements of the pagerank vector with the contents of the third column of the hyperlink matrix will similarly give us , the pagerank of page 3:

你可以验证一下，使用超链接矩阵的第四列和第五列，我们分别会得到和。这个运算——将 PageRank 向量元素与超链接矩阵每一列内容的乘积相加——实际上是将 PageRank 向量与超链接矩阵相乘。

You can verify that using the fourth and fifth columns of the hyperlink matrix we’ll get and , respectively. This operation—of summing the products of the elements of the pagerank vector with the contents of each column of the hyperlink matrix—is actually the product of the pagerank vector with the hyperlink matrix.

除非你熟悉矩阵运算，否则这可能会令人困惑，因为我们通常讨论的是两个数的乘积，也就是常见的乘法，而不是向量和矩阵等结构之间的乘积。我们可以对其他实体（不仅仅是数字）定义数学运算，只要它适合我们。向量与矩阵的乘积就是这样一种运算。它本身并没有什么神秘之处：它只是我们定义的一种涉及向量和矩阵元素的特定计算。

Unless you are familiar with matrix operations, this may be confusing because we usually talk about the product of two numbers, which is the common multiplication, and not about the product of constructs like vectors and matrices. We can define mathematical operations on other entities, not just numbers, as long as it suits us. The product of a vector with a matrix is such an operation. There is no mystery involved in it: it is simply an operation that we define as a particular calculation involving the elements of the vector and matrix.

假设我们制作百吉饼和羊角面包，售价分别为 2.00 美元和 1.50 美元。我们有两家商店；某一天，第一家商店售出 10 个百吉饼和 20 个羊角面包，而第二家商店售出 15 个百吉饼和 10 个羊角面包。如何计算每家商店的总销售额？

Suppose that we make bagels and croissants that we sell for $2.00 and $1.50, respectively. We have two shops; on a particular day, the first shop sells 10 bagels and 20 croissants, while the second shop sells 15 bagels and 10 croissants. How do we find the total sales per shop?

为了找到第一家商店的总销售额，我们将百吉饼的价格乘以该商店销售的百吉饼数量，将羊角面包的价格乘以该商店销售的羊角面包数量，然后将这两个价格相加：

To find the total sales from the first shop, we will multiply the price of a bagel with the number of bagels sold in that shop, and the price of a croissant with the number of croissants sold there, and we’ll add these two:

我们将采取同样的方式，找出第二家商店的总销售额：

We’ll do the same thing to find the total sales from the second shop:

为了更简洁地表达这一点，我们将百吉饼和羊角面包的价格写成一个向量：

To express this more succinctly, we write down the prices for the bagels and croissants as a vector:

我们还将每日销售额写成一个矩阵。该矩阵有两列，每家商店一列；还有两行，一行代表百吉饼，一行代表羊角面包：

We also write down the daily sales in a matrix. The matrix will have two columns, one per shop, and two rows, one for the bagels and one for the croissants:

然后，为了计算每家商店的总销售额，我们将向量的元素与销售额矩阵的每一列相乘，然后相加。这定义了向量与矩阵的乘积：

Then to find the total sales per shop, we multiply the elements of the vector with each column of the sales matrix and add them up. This defines the product of the vector with the matrix:

向量与矩阵的乘积是两个矩阵乘积的一个特例。让我们扩展一下这个例子，这样就不用再用一个向量来表示商品的价格了百吉饼和羊角面包，我们有一个矩阵，其中包含每次销售的价格和利润：

The product of a vector with a matrix is a special case of the product of two matrices. Let’s extend the example so that instead of having a vector with the prices of the bagels and croissants, we have a matrix with the prices and profits per sale:

为了计算每家商店的总销售额和每家商店的总利润，我们将创建一个矩阵，其中第i行第j列的元素是价格和利润矩阵第 i 行与销售额矩阵第 j行乘积之和。这两个矩阵乘积的定义如下：

To find the total sales per shop and total profit per shop, we will create a matrix in which the entries in the ith row and jth column will be the sum of products of the ith row of the prices and profits matrix with the jth row of the sales matrix. This is the definition of the product of the two matrices:

回到 PageRank 的问题，每一轮 PageRank 向量的计算实际上是将上一轮 PageRank 向量的值与超链接矩阵相乘。随着轮次的推进，我们会得到连续的 PageRank 估计值，也就是由这些值组成的 PageRank 向量的连续估计值。为了获得这些连续的 PageRank 向量估计值，我们只需要将每一轮的向量与超链接矩阵相乘，从而得到下一轮的向量。

Returning to pagerank, in each round the calculation of the pagerank vector is really the product of the value of the pagerank vector in the previous round with the hyperlink matrix. As we go through the rounds, we get successive estimates of the pageranks—that is, successive estimates of the pagerank vector that is made up of them. To get these successive estimates of the pagerank vector we only need to multiply the vector in each round with the hyperlink matrix, thereby getting the vector for the next round.

在第一轮中，我们从一个 PageRank 向量开始，其内容均为 ，其中n是页面数。如果我们将第一个 PageRank 向量表示为，将第一轮结束时的 PageRank 向量表示为，将超链接矩阵表示为H，则有：

In the first round, we start with a pagerank vector whose contents are all equal to where n is the number of pages. If we denote this first pagerank vector by , the pagerank vector at the end of the first round by , and the hyperlink matrix by H, we have:

在每一轮中，我们都使用该轮的 PageRank 向量来计算下一轮的 PageRank 向量。在第二轮投票中，我们得到了第三个 PageRank 估值，也就是第三个 PageRank 向量，我们进行了如下计算：

In each round we use the pagerank vector of that round to calculate the pagerank vector for the following round. In the second voting round, where we got our third pagerank estimates—that is, our third pagerank vector—we performed the calculation:

在第三轮投票中，我们得到了第四个 PageRank 向量：

In the third voting round, we got our fourth pagerank vector:

与每次迭代一样，我们将前一次迭代的结果乘以超链接矩阵，最后这是一系列PageRank向量的逐次估计值与超链接矩阵的乘积。如我们所见，这相当于将初始PageRank向量乘以超链接矩阵的幂。这种计算逐次近似值的方法称为幂法。因此，我们看到，计算一组网页的PageRank就是将幂法应用于PageRank向量和超链接矩阵，直到得到的PageRank向量不再变化，或者说，直到它收敛到一个稳定的值——也就是我们最终的PageRank指标。

As in every iteration, we multiply the result of the previous iteration by the hyperlink matrix, and in the end this is a series of products of the successive estimates of the pagerank vector by the hyperlink matrix. As we see, this is equivalent to multiplying the initial pagerank vector with increasing powers of the hyperlink matrix. This method of calculating successive approximations is called the power method. We see therefore that the calculation of the pageranks of a set of web pages is an application of the power method to the pagerank vector and hyperlink matrix, until the resulting pagerank vector does not change, or as we say, until it converges to a stable value—our final pagerank metrics.

我们刚刚对如何计算网络图的 PageRank 给出了更精确的描述：

We have just reached a more precise description of how to calculate the pageranks of a web graph:

1. 1. 形成图的超链接矩阵。
2. 1. Form the hyperlink matrix of the graph.
3. 2. 从初始 PageRank 估计开始，为每个页面赋予一个 PageRank，其中n是页面总数。
4. 2. Start with initial pagerank estimates, giving a pagerank of to each page, where n is the total number of pages.
5. 3. 应用幂法，将 pagerank 向量乘以超链接矩阵，直到 pagerank 向量的值收敛。
6. 3. Apply the power method, multiplying the pagerank vector by the hyperlink matrix until the values of the pagerank vector converge.

这种表述不仅简洁，还能让我们将问题转移到线性代数领域，线性代数是数学中处理矩阵及其运算的分支。目前已有一套完善的理论体系我们可以用它来研究幂法以及矩阵运算的高性能实现，例如我们之前描述的乘法。该问题的矩阵形式也有助于研究幂法是否总是收敛，以便我们总能找到图的 PageRank 解。

Apart from being succinct, this formulation allows us to transfer the problem to the realm of linear algebra, the branch of mathematics that treats matrices and operations on them. There is a well-established body of theory that we can use to investigate the power method as well as performant implementations of matrix operations, such as the multiplication that we described. The matrix formulation of the problem will also help investigate whether the power method will always converge so that we can always come up with a solution to the pageranks of a graph.

悬垂节点和随机冲浪者

Dangling Nodes and the Random Surfer

现在我们来看一个更简单的图的例子，它仅由三个节点组成：

We now turn to an example of a simpler graph, consisting of just three nodes:

我们想求出这三个节点的 PageRank。我们遵循相同的算法。我们将 PageRank 向量初始化为，赋予所有节点相同的 PageRank。然后将 PageRank 向量乘以超链接矩阵，即：

We want to find the pageranks of these three nodes. We follow the same algorithm. We initialize the pagerank vector to , giving equal pageranks to all nodes. Then we multiply the pagerank vector with the hyperlink matrix, which is:

如果我们开始幂方法的迭代，将 PageRank 向量与超链接矩阵相乘以更新 PageRank 向量，然后一次又一次地进行，我们会发现经过四次迭代后，所有的 PageRank 都下降到了零：

If we start the iterations of the power method, multiplying the pagerank vector with the hyperlink matrix to update the pagerank vector, and then again and again, we’ll find out that after four iterations, all pageranks have gone down to zero:

这显然是个问题。我们并不期望所有页面的重要性都为零。毕竟，页面 3 有两个反向链接，页面 2 只有一个反向链接，所以我们自然会期望这些反向链接显示在结果中，更不用说我们还希望所有页面的 PageRank 总和为 1 了。结果，所有页面的重要性都变得毫无意义。

That is clearly a problem. We do not expect all pages to have zero importance here. After all, page 3 has two backlinks and page 2 has one backlink, so somehow we would expect this to show on the results, let alone the fact that we also want the total sum of the pageranks to be one. Here nothing ended up being of any import at all.

问题的根源在于节点 3。虽然该节点有反向链接，因此会提升重要性，但它没有外链。因此，它在某种程度上吸收了图谱其余部分的重要性，但却没有将其重新分配到任何地方。它就像一个自私的节点或黑洞：进去的不会出去。经过几次迭代后，它就像一个汇聚点，所有 PageRank 值都流入其中，然后消失殆尽。

The cause of the problem is node 3. Although this node has backlinks and would thereby gain importance, it has no outgoing links. So in a way it sucks importance from the rest of the graph, but does not redistribute it anywhere. It acts as a selfish node or black hole: what goes in, does not go out. After a few iterations, it has acted as a sink where all pagerank values have gone in and vanished.

这类节点被称为悬垂节点，因为它们悬挂在图的（死）端。在网络上，没有什么可以阻止这类页面的存在。虽然网页通常同时包含传入和传出链接，但没有传出链接的页面也可能出现，并且会严重破坏我们之前描述的幂律方法。

Such nodes are called dangling nodes because they hang at the (dead) ends of the graph. On the web, nothing prohibits the existence of such pages. Although web pages usually have both incoming and outgoing links, a page with no outgoing links can appear and would wreak havoc with the power method as we have described it.

为了解决这个问题，我们用了一个比喻。我们想象一个人在网上冲浪，从一个页面跳转到另一个页面。要从一个页面转到另一个页面，冲浪者通常会点击链接。但是，冲浪者会遇到一个悬挂节点：一个没有链接到任何其他页面的页面。我们不希望冲浪者被困在那里，所以我们赋予冲浪者跳转到网络上任何页面的能力。这就像我们在网上冲浪，从一个页面跳转到另一个页面，直到到达死胡同。即使到达死胡同，我们也不会放弃。我们随时可以在网络浏览器中输入另一个地址，然后转到我们想要的任何其他网页，即使悬挂页面上没有指向该网页的链接。这就是我们希望冲浪者做的事情。当冲浪者不知道该去哪里时，他会从网络上选择一个页面，任意一个页面，然后继续冲浪。冲浪者成为一名随机冲浪者，配备传送装置，可以立即将冲浪者带到任何地方。

To overcome the problem, we work with a metaphor. We imagine that we have a human who surfs the web, jumping from page to page. To go from one page to another, the surfer normally follows a link. But then the surfer comes on a dangling node: a page with no links to any other page. We don’t want our surfer to remain trapped in there so we give the surfer the capability to jump to any other page, anywhere on the web. It is as if we are surfing the web from page to page until we reach a dead end. When we get there, we don’t give up and stop. We can always type another address in our web browser and move to any other web page we want, even if no links exist to it from the dangling page. This is what we want our surfer to do. When at a loss about where to go, the surfer will pick a page, any page, from the web and go there to continue surfing. The surfer becomes a random surfer, equipped with a teleportation device that can take the surfer instantly to any place at all.

把这个比喻带回到 PageRank 上，我们将超链接矩阵解释为，它给出了浏览者点击链接到达特定页面的概率。在我们的以三节点为例，超链接矩阵的第一行告诉我们，当浏览者在第1页时，选择第2页或第3页的概率相同。第二行告诉我们，当浏览者在第2页时，他总是会选择访问第3页。回到第一个例子，如果浏览者访问的是第5页，那么他有可能访问第2页、第3页或第4页，每种情况的概率均为。

To take this metaphor back to pagerank, we interpret the hyperlink matrix as giving us the probabilities that a surfer will follow a link to go to a particular page. In our three-nodes example, the first row of the hyperlink matrix tells us that when on page 1, the surfer will choose either page 2 or 3 with equal probability. The second row tells us that when on page 2, the surfer will always choose to visit page 3. Going back to our first example for a moment, if the surfer lands on page 5, then it is possible to go to page 2, 3, or 4 with a probability of for each of these outcomes.

悬挂节点表现为一行全为零。这样一来，浏览者就没有任何可能去任何地方。这时，随机浏览者就发挥作用了。正如我们所说，该浏览者会跳转到图中的任何页面。这意味着，实际上，我们改变了超链接矩阵，使其不再包含全为零的行。由于我们希望浏览者以相同的概率跳转到任何网页，因此我们将用（在我们的示例中为 ）而不是零来填充该行。我们的矩阵将变为：

A dangling node manifests itself in the presence of a row full of zeros. Then there is no probability that the surfer will go anywhere. This is where the random surfer kicks in. As we said, that surfer will jump to any page in the graph. That means that in effect, we change the hyperlink matrix so that it no longer has rows with zeros. As we want the surfer to jump to any web page with equal probability, instead of zeros we’ll fill the row with , or in our example, . Our matrix will become:

现在，到达第 3 页的浏览者可以以相同的概率访问图表中的任意页面。浏览者甚至可能暂时停留在同一页面上，但这无关紧要，因为浏览者可以反复尝试，最终会随机选择一个不同的目标页面。我们将这个修改后的超链接矩阵称为S矩阵，其中我们将零行改为值等于 的行。如果我们使用S矩阵运行幂函数，则 PageRank 的演变将是：

Now the surfer who lands on page 3 can go to any page in the graph with equal probability. The surfer may even stay temporarily on the same page, but that does not matter, as the surfer can try again and again, and at some point a different target page will be selected at random. We call this modified hyperlink matrix, where we change zero rows to rows with values equal to , the S matrix. If we run the power method using the S matrix, then the evolution of the pageranks will be:

这次算法收敛到了非零值；没有发生重要性被吸走的情况。而且，结果也合情合理。PageRank最高的是页面3，它有两个反向链接；其次是页面2，有一个反向链接；然后是页面1，它完全没有反向链接。

This time the algorithm converges to nonzero values; no sucking out of importance occurs. Also, the results make sense. The highest pagerank is achieved by page 3, which has two backlinks; then comes page 2, with one backlink, and then page 1, which has no backlinks at all.

谷歌矩阵

The Google Matrix

我们似乎已经解决了这个问题，但在更复杂的情况下，类似的问题仍然会出现。下图没有悬垂节点：

We seem to have solved the problem, but a similar issue raises its head in more complex situations. The following graph has no dangling nodes:

如果我们运行该算法，我们会发现两个节点（页面 1 和页面 4）最终的 PageRank 为零：

If we run the algorithm, we find that two nodes, pages 1 and 4, end up with zero pagerank:

实际情况是，即使没有悬挂节点，也有一组节点充当图其余部分的汇聚点。仔细观察该图，你会发现节点 2、3、5 和 6 作为一个组，只包含传入链接。虽然可以从节点 1 或 4 进入该组，但一旦进入，就只能在组内移动，无法离开。我们的随机浏览者将被困住，而不是被困在一个网页内。时间，但在一组仅在它们之间链接的页面内。

What happened is that even though there is no dangling node, there is a set of nodes that act as a sink for the rest of the graph. If you scrutinize the graph, you will see that the nodes 2, 3, 5, and 6, taken together as a group, have only incoming links. It is possible to go from node 1 or 4 to this group, but once we are in, we can only move inside the group. We are not able to go outside. Our random surfer will be trapped, not inside a single web page this time, but inside a group of pages that link only between themselves.

我们再次需要帮助随机浏览者摆脱这个陷阱。这次的解决方案需要对超链接矩阵进行更全面的修改。我们最初的超链接矩阵允许浏览者仅使用原始图中现有的链接从一个页面跳转到另一个页面。然后，我们修改了超链接矩阵，使其能够处理元素全为零的行，并提出了一个S矩阵，使浏览者能够摆脱悬空节点。这使得随机浏览者在处于悬空节点时能够跳转到图中的任意位置。现在，我们将通过修改S矩阵进一步改变随机浏览者的行为。

We need again to help the random surfer escape from this trap. This time the solution requires more comprehensive changes to the hyperlink matrix. Our initial hyperlink matrix allowed the surfer to go from page to page only using the existing links in the original graph. Then we modified the hyperlink matrix to handle rows with all zero elements and came up with the S matrix that allowed the surfer to get away from dangling nodes. This enabled the random surfer to jump to anywhere in the graph when in a dangling node. Now we will change the behavior of the random surfer a bit more by modifying the S matrix.

现在，当冲浪者落在一个节点上时，可能的移动方式就是S矩阵所指示的。在最后例如，S矩阵与超链接矩阵相同，因为不存在零行：

Right now, when a surfer lands on a node, the possible moves are those indicated by the S matrix. In the last example, the S matrix is the same as the hyperlink matrix because no zero rows exist:

如果随机冲浪者落在第 5 页，那么可能的移动方向是第 2、3 或 6 页，所有方向都有概率，正如S矩阵所示。我们将使随机冲浪者更加灵活，使其能够按照S矩阵移动，虽然不一定是，但概率是，我们将选择a；这样，随机冲浪者就会以 的概率跳到图中的任意位置，不受S矩阵的约束。

If the random surfer lands on page 5, then the possible moves are to pages 2, 3, or 6, all with probability, as the S matrix indicates. We will make the random surfer more agile, with the power to move following the S matrix not always, but with some probability a that we will choose; then for some probability , the random surfer will jump anywhere in the graph, unconstrained by the S matrix.

能够从图中的任意位置跳转到任意位置意味着矩阵中不能有任何零——因为零元素表示无法进行移动。为了实现我们想要的效果，我们需要将一行中的零元素增加某个值，并减少非零元素，使得整行元素的总和始终为 1。矩阵的最终精确值可以通过线性代数计算得出，基于S和概率a。由此导出的新矩阵称为Google 矩阵，我们用符号G来表示。如果随机冲浪者的行为由 Google 矩阵决定，那么它就会按照我们的预期运行：冲浪者似乎以概率a遵循S矩阵，并以概率 独立移动。在我们的示例中，Google 矩阵为：

The ability to jump from anywhere to anywhere in the graph means that we cannot have any zeros at all in the matrix—because a zero entry denotes a move that cannot be made. To achieve what we want, we will need to increase the zero entries in a row by some value and decrease the nonzero entries so that the whole row always sums up to one. The exact final values of the matrix can be calculated through linear algebra, based on S and the probability a. The new matrix that will be derived is called the Google matrix, and we use the symbol G. If the behavior of the random surfer is determined by the Google matrix, it will work out as we want: the surfer will appear to be following the S matrix with probability a and move independently with probability . In our example, the Google matrix is:

将其与S矩阵进行比较。观察第一行，我们有两个元素为，其余元素为零。现在在 Google 矩阵中，我们将两个元素转换为，其余元素从 0 转换为。其他行也发生了类似的变换。那么，如果随机浏览者浏览的是第 1 页，那么他可能转到第 2 页和第 5 页，概率为对于其中任何一个，或者对于其中每一个都有概率的任何其他页面。

Compare that to the S matrix. Observe that in the first row, we had two entries with and the rest were zero. Now in the Google matrix, we have the two entries turned to , and the rest of the entries turned from 0 to . Similar transformations have occurred in the other rows. If, then, the random surfer lands on page 1, the possible moves out are to pages 2 and 5 with probability for either of them, or any other page with probability for each one of them.

现在我们可以给出 PageRank 算法的最终定义：

We are now able to give the final definition of the PageRank algorithm:

1. 1. 形成图的Google矩阵。
2. 1. Form the Google matrix of the graph.
3. 2. 从初始 PageRank 估计开始，为每个页面赋予一个 PageRank，其中n是页面总数。
4. 2. Start with initial pagerank estimates, giving a pagerank of to each page, where n is the total number of pages.
5. 3. 应用幂法，将 pagerank 向量乘以 Google 矩阵，直到 pagerank 向量的值收敛。
6. 3. Apply the power method, multiplying the pagerank vector by the Google matrix until the values of the pagerank vector converge.

我们简单地用“Google矩阵”替换了初始算法中的“超链接矩阵”。如果我们在图中追踪该算法，并将其与一组接收器节点关联起来，我们将得到：

We simply substituted “Google matrix” for “hyperlink matrix” of the initial algorithm. If we trace this algorithm in our graph with the group of sink nodes, we’ll get:

效果很好；我们不再得到零页面排名。

It works out fine; we get no zero pageranks anymore.

谷歌矩阵的幂法总是有效的。线性代数告诉我们，它总是会收敛到一组最终的 PageRank 值，这些值的和为 1，而不会出现悬挂节点或图的某些部分耗尽其余部分的 PageRank 值的情况。我们甚至不需要将 PageRank 值初始化为初始值的准确值。任何初始值都可以，只要它们和为 1。

The power method with the Google matrix will work always. Linear algebra tells us that it will always converge to a final set of pagerank values, the sum of which will be one, without suffering from dangling nodes or parts of the graph draining the pageranks of the rest of the graph. We don’t even need to initialize the pageranks to exactly when we start. Any initial set of values will do, as long as they sum up to one.

PageRank实践

PageRank in Practice

我们已经确定有一种方法可以在任何图表中找到页面排名，但问题仍然是结果最终是否合理。

Having established that we have a method to find the pageranks in any graph, the question remains whether the results are in the end sensible.

按照我们定义的方式，PageRank 向量相对于 Google 矩阵来说是一个特殊的向量。幂函数完成后，PageRank 向量不再变化。因此，如果我们将 Google 矩阵乘以 PageRank 向量，我们得到的就是相同的 PageRank 向量。在线性代数中，这个向量被称为Google 矩阵的第一个特征向量。无需深入数学，其底层理论支持这个向量对矩阵具有某种特殊意义的观点。

The pagerank vector, in the way that we have defined it, is a special vector in relation to the Google matrix. When the power method finishes, the pagerank vector does not change any more. Therefore if we multiply the Google matrix by the pagerank vector we will get simply the same pagerank vector. In linear algebra, this vector is called the first eigenvector of the Google matrix. Without going deep into the mathematics, the underlying theory supports the notion that this vector has some special significance to the matrix.

除了数学之外，PageRank 是否是衡量网页重要性的好方法的最终决定因素是其结果对我们人类的实用性。谷歌搜索引擎给出的结果很好，这意味着结果符合我们（搜索引擎用户）认为重要的内容。如果PageRank向量只是一种数学上的奇思妙想，与网页的重要性无关，我们今天就不会关注它了。

Beyond mathematics, the final arbiter of whether PageRank is a good way to assign importance to web pages is the utility of its results to us humans. The Google search engine gives good results, meaning that the results are in accordance with what we, the users of the search engine, regard as being important. If the pagerank vector was a mathematical curiosity that bore no relation to the significance of web pages, we would not be concerned with it today.

PageRank 的另一个优势是它可以高效地实现。Google 矩阵非常庞大；我们需要为网络上的每个页面分配一行和一列。然而，正如我们所见，Google 矩阵是从S矩阵派生而来的，而 S 矩阵又从超链接矩阵派生而来。我们实际上并不需要创建和存储 Google 矩阵本身；我们可以通过对超链接矩阵进行矩阵运算来动态创建它。这很方便。与 Google 矩阵中没有零元素相比，超链接矩阵中有很多零元素。网络可能有数十亿个页面，但每个页面都只链接到少数其他网页。超链接矩阵就是我们所说的稀疏矩阵：它大部分都是零，只有一些非零元素，这些非零元素的数量级比零元素要小。因此，我们可以使用巧妙的技巧来存储矩阵，这样就不需要占用大量内存来填充大部分零元素和少量非零元素，而是只存储非零元素出现的位置。与其存储在整个超链接矩阵中，我们只需要存储非零元素的坐标，这只需要很小一部分存储空间。这在 PageRank 算法的实际实现中为我们提供了巨大的优势。

An additional advantage of PageRank is that it can be implemented efficiently. The Google matrix is huge; we need one row and one column for every single page on the web. Yet the Google matrix is derived, as we saw, from the S matrix, which in turn is derived from the hyperlink matrix. We do not really need to create and store the Google matrix itself; we can create it dynamically with matrix operations on the hyperlink matrix. This is convenient. In contrast to the Google matrix, which has no zeros anywhere, the hyperlink matrix has lots and lots of zeros. The web may have billions of pages, but every single page links to only a small number of other web pages. The hyperlink matrix is what we call a sparse matrix: one that is mostly full of zeros, with only some nonzero entries, which are scales of magnitude fewer than the zero entries. Thus we can store the matrix using clever techniques that instead of requiring a big slab of memory to fill with mostly zeros and a few nonzeros, store only the positions where the nonzeros occur. Rather than storing the whole hyperlink matrix, we need only store the coordinates of the nonzero entries, which will require only a fraction of the storage space. This gives us big leverage in the practical implementations of the PageRank algorithm.

最后，需要注意的是：虽然我们知道 PageRank 在 Google 的成功中发挥了至关重要的作用，但我们并不清楚 PageRank 在 Google 中是如何应用的，甚至是否应用。Google 搜索引擎多年来一直在不断发展，但这些变化并未公开。我们知道 Google 会利用我们过去的搜索记录来优化搜索结果。它可以根据我们所在的国家/地区调整搜索结果，还可以参考全球其他用户搜索的整体趋势。所有这些都是 Google 用来改进产品并在搜索引擎领域保持领先地位的秘诀。然而，这并不会降低算法在解决网页排名问题（以图中的节点表示）时的效率。1

Finally, an important caveat. Although we know that PageRank played a crucial role in the success of Google, we do not know how, or even if, PageRank is used in Google today. The Google search engine has been evolving during the years, and the changes are not made public. We know that Google uses our past searches to fine-tune the results that it presents to our queries. It can tune the results depending on the country that we live in. It can also take into account the overall trends in the queries that other people make all around the world. All these are part of the secret sauce that Google uses to improve its product and retain its position in the search engine business against competitors. This, however, does not detract from the algorithm’s efficiency in solving the problem of ranking web pages, represented as nodes in a graph.1

PageRank 凸显了算法的另一个方面。算法的成功不仅仅取决于其优雅和效率，还取决于算法如何映射到问题上。这是一种创造性的行为。要解决网络搜索问题，必须克服网络规模庞大这一难题。但一旦你如果将网络视为一个图，它的大小就变成了优势，而不是障碍。正是因为有如此多的页面相互链接，你才可能期望基于图的链接结构的方法最终能够奏效。找到问题建模的方法是找到用算法解决问题的第一步。

PageRank highlights an additional aspect of algorithms. The success of an algorithm does not hinge only on its elegance and efficiency. It also has to do with the mapping of the algorithm to a problem. This is a creative act. To solve the problem of web search, one has to overcome the issue of the sheer size of the web. But once you conceive of the web as a graph, its size turns into an advantage, not a hindrance. It is exactly because there are so many pages, hyperlinked to each other, that you may expect that a method that is based on the link structure of the graph will in the end work. Finding the way to model a problem is the first step in finding the way to solve it with an algorithm.

真实神经元与人工神经元

Neurons, Real and Artificial

我们的出发点将是深度学习系统的主要构建模块，它确实源自生物学。大脑是神经系统的一部分，而神经系统的主要组成部分是被称为神经元的细胞。神经元具有特殊的形状；它们看起来与球状的我们通常与细胞联系在一起的结构。下面是最早的神经元图像之一，由现代神经科学的奠基人、西班牙人圣地亚哥·拉蒙·卡哈尔于1899年绘制。1

Our starting point will be the main building block of deep learning systems, which does come from biology. The brain is part of the nervous system, and the main components of the nervous system are cells called neurons. Neurons have a particular shape; they look different from the globular structures that we usually associate with cells. You can see below one of the first images of neurons, drawn in 1899 by the Spanish Santiago Ramón y Cajal, a founder of modern neuroscience.1

图像中间突出的两个结构是鸽子大脑的两个神经元。如你所见，神经元由细胞体和从中伸出的细丝组成。这些细丝通过突触将神经元连接到其他神经元，从而将神经元嵌入网络中。神经元是不对称的。每个神经元的一侧有许多细丝，另一侧只有一根细丝。我们可以将一侧的许多细丝视为神经元的输入，将另一侧的长向外细丝视为神经元的输出。神经元从其传入突触中获取电信号形式的输入，并可能向其他神经元发送信号。它接收的输入越多，输出信号的可能性就越大。我们说神经元随后被激发或激活。

The two structures that stand out in the middle of the image are two neurons of the pigeon brain. As you can see, a neuron consists of a cell body and the filaments that extrude from it. These filaments connect a neuron to other neurons through synapses, embedding the neurons in a network. The neurons are asymmetrical. There are many filaments on the one side and one filament on the other side of each neuron. We can think of the many filaments on the one side as the neuron’s inputs, and the long outgoing filament on the other side as the neuron’s output. The neuron takes input in the form of electric signals from its incoming synapses and may send a signal to other neurons. The more inputs it receives, the more likely it is to output a signal. We say that the neuron then fires or is activated.

人脑是一个庞大的神经元网络，数量约为一千亿，每个神经元平均与数千个其他神经元相连。我们目前还没有办法构建类似的东西，但我们可以用简化、理想化的神经元模型来构建系统。这是一个人工神经元的模型：

The human brain is a vast network of neurons, which number about one hundred billion, and each one of them is connected on average to thousands of other neurons. We do not have the means to build anything like that, but we can build systems out of simplified, idealized models of neurons. This is a model of an artificial neuron:

那是生物神经元的抽象版本，仅仅是具有多个输入和一个输出的结构。生物神经元的输出取决于其输入；同样，我们希望人工神经元根据其输入进行激活。我们不在大脑生物化学的领域，而是在计算的世界中，因此我们需要为我们的人工神经元建立一个计算模型。我们假设神经元接收和发送的信号是数字。然后，人工神经元获取所有输入，基于它们计算一些算术值，并在其输出上产生一些结果。我们不需要任何特殊电路来实现人工神经元。你可以将它想象成计算机内的一个小程序，它获取输入并产生输出，就像任何其他计算机程序一样。我们不需要真正构建人工神经网络；我们可以而且确实在模拟它们。

That is an abstract version of a biological neuron, being just a structure with a number of inputs and one output. The output of a biological neuron depends on its input; similarly, we want the artificial neuron to be activated depending on its input. We are not in the realm of brain biochemistry, but in the world of computing, so we need a computational model for our artificial neuron. We assume that the signals received and sent by neurons are numbers. Then the artificial neuron takes all its inputs, calculates some arithmetic value based on them, and produces some result on its output. We do not need any special circuit for implementing an artificial neuron. You can think of it as a small program inside a computer that takes its inputs and produces an output, much like any other computer program. We do not need to build artificial neural networks literally; we can and do simulate them.

生物神经网络学习过程的一部分是神经元之间突触的增强或减弱。获得新的认知能力和吸收知识会导致一些神经元之间的突触增强，而另一些突触则减弱甚至完全消失。此外，突触不仅可以​​激发神经元的放电，还可以抑制其激活；当信号到达该突触时，该神经元不应该放电。婴儿实际上拥有更多的突触。他们的大脑比成年人的大脑更发达。成长的一部分就是修剪我们大脑中的神经网络。或许我们可以把婴儿的大脑想象成一块大理石；随着我们人生岁月的流逝，这块大理石会被我们的经历和所学的东西凿刻，最终形成一个完整的形状。

Part of the learning process in biological neural networks is the strengthening or weakening of the synapses between neurons. The acquisition of new cognitive abilities and absorption of knowledge result in some synapses between neurons getting stronger, while others get weaker or even drop off completely. Moreover, synapses may not only excite a neuron to fire but also inhibit its activation; when a signal arrives on that synapse, the neuron should not fire. Babies have actually more synapses in their brains than adults. Part of growing up is pruning the neural network inside our heads. Perhaps we could think of the infant brain as a block of marble; as we go through the years in our lives, the block is chipped through our experiences and the things we learn, and a form emerges.

在人工神经元中，我们通过对输入应用权重来近似突触的可塑性及其兴奋或抑制作用。在我们的模型人工神经元中，我们有n 个输入，，，...，。我们对每个输入应用一个权重，，，...，。每个权重乘以相应的输入。神经元收到的最终输入是乘积之和： 。我们为这个加权输入添加一个偏差 b，你可以将其视为神经元激发的倾向；偏差越高，神经元被激活的可能性就越大，而添加到加权输入的负偏差实际上会抑制神经元激发。

In an artificial neuron, we approximate the plasticity of synapses, their excitatory or inhibitory role, through weights we apply to the inputs. In our model artificial neuron, we have n inputs, , , . . . , . To each one of them we apply a weight, , , . . . , . Each weight is multiplied by the corresponding input. That final input received by a neuron is the sum of the products: . To this weighted input we add a bias b, which you can think of as the propensity the neuron has to fire; the higher the bias, the more likely it is to be activated, while a negative bias added to the weighted input will actually inhibit the neuron from firing.

权重和偏差是神经元的参数，因为它们会影响神经元的行为。正如生物神经元的输出取决于其输入一样，人工神经元的输出也取决于其获得的输入。这通过将输入馈入一个特殊的激活函数来实现，该函数的结果就是神经元的输出。下图展示了使用激活函数的示意图：

The weights and bias are the parameters of the neuron because they influence its behavior. As the output of a biological neuron depends on its inputs, so the output of an artificial neuron depends on the input it gets. This happens by feeding the input into a special activation function, the result of which is the neuron’s output. This is what happens, diagrammatically, using as a stand-in for the activation function:

最简单的激活函数是阶跃函数，它给出的结果为 0 或 1。如果激活函数的输入大于 0，则神经元触发并输出 1，否则保持沉默并输出 0：

The simplest activation function is a step function, giving us a result of 0 or 1. The neuron fires and outputs 1 if the input to the activation function is greater than 0, or stays silent outputting 0 otherwise:

与其考虑偏差，不如考虑阈值。如果加权输入超过阈值，否则输出 0。事实上，如果我们将神经元的行为写成公式，第一个条件是或。利用，我们得到，其中t，即偏差的反函数，是加权输入需要通过的阈值，才能使神经元触发。

Instead of a bias, it is helpful to think of a threshold. The neuron outputs 1 if the weighted input exceeds a threshold or outputs 0 otherwise. Indeed, if we write the behavior of the neuron as a formula, the first condition is or . By using , we get , where t, the opposite of the bias, is the threshold that the weighted input needs to pass for the neuron to fire.

在实践中，我们倾向于使用其他相关的激活函数来代替阶跃函数。下一页你将看到三种常见的激活函数。

In practice we tend to use other, related activation functions instead of the step function. On the next page you can see three common ones.

顶部的函数称为sigmoid ，因为它具有 S 形。2其输出范围从 0 到 1。较大的正输入会导致输出接近于 1；较大的负输入会导致输出接近于 0。这近似于生物神经元，它在输入较大时激发，否则保持沉默，并且是阶跃函数的平滑近似值。中间的激活函数称为tanh ，是双曲正切的缩写（有多种发音方式：“tan-H”、“then”或带有软 th 的“thents”，如感谢）。3它看起来像 sigmoid 函数，但不同之处在于它的输出范围从到；较大的负输入会导致负输出，模仿抑制信号。底部的函数称为整流器；它将所有负输入变为 0，否则其输出与输入成正比。下表显示了三个激活函数对不同输入的输出。

The one on the top is called sigmoid because it has an S shape.2 Its output ranges from 0 to 1. A large positive input results in outputs close to 1; a large negative input results in an output close to 0. This approximates a biological neuron that fires on large inputs and stays silent otherwise, and is a smooth approximation to the step function. The activation function in the middle is called tanh, short for hyperbolic tangent (there are various ways to pronounce it: “tan-H,” “then,” or “thents” with a soft th, as in thanks).3 It looks like the sigmoid function, but it differs in that its output ranges from to ; a large negative input results in a negative output, mimicking an inhibitory signal. The function at the bottom is called a rectifier; it turns all negative inputs to 0, otherwise its output is directly proportional to its input. The following table shows the output of the three activation functions for different inputs.

如果你想知道为什么激活函数（当然还有其他函数）会如此繁多，那是因为实践证明，某些激活函数在某些应用中比其他激活函数更合适。由于激活函数对神经元的行为至关重要，因此神经元通常以其激活函数命名。使用阶跃函数的神经元被称为感知器。此外，还有 S 型和 tanh 型神经元。我们也将神经元称为单元，而使用整流器的神经元被称为ReLU（即整流线性单元）。

If you wonder why the proliferation of activation functions (there are also others), it is because it has been found in practice that particular activation functions are more suitable in some applications than others. As the activation function is crucial for the behavior of a neuron, neurons are often named by their activation functions. A neuron that uses the step function is called a Perceptron.4 Then we have sigmoid and tanh neurons. We also call neurons units, and a neuron using the rectifier is called a ReLU, for rectified linear unit.

单个人工神经元就能学会区分两组事物。例如，以下一页上图的数据为例，该图描绘了一组观测值，横轴为 ，纵轴为 。我们希望构建一个能够区分这两个色块的系统。给定任何物品，系统都能判断该物品属于哪一类。实际上，它会创建一个决策边界，就像下图所示。对于 的任何组合，系统都能告诉我们该物品属于较亮的组还是较暗的组。

A single artificial neuron can learn to distinguish between two sets of things. For example, take the data in the figure on the top of the next page, portraying a set of observations with two features, , on the horizontal axis, and , on the vertical axis. We want to build a system that will tell apart the two blobs. Given any item, the system will be able to decide whether the item falls in one group or another. In effect, it will create a decision boundary, like in the figure at the bottom. For any combination of , it will tell us whether the item belongs to the lighter or darker group.

神经元只有两个输入。它会接收每一对输入并计算输出。如果我们使用 S 型激活函数，输出将在 0 到 1 之间。我们将大于 0.5 的值归入一类，将其他值归入另一类。这样，神经元将充当分类器，将数据分类到不同的类别中。但它是如何做到这一点的呢？神经元如何才能达到对数据进行分类的程度呢？

The neuron will have only two inputs. It will take each pair and calculate an output. If we are using the sigmoid activation function, the output will be between 0 and 1. We’ll take the values greater than 0.5 to fall into one group and the other values to fall into the other. In this way the neuron will act as a classifier, sorting our data into distinct classes. But how does it do that? How can the neuron get to the point of being able to classify data?

学习过程

The Learning Process

我们的神经元在诞生之初无法识别任何类型的数据；它通过学习来识别它们。它学习的方式是通过示例。整个过程类似于让学生学习，我们给他们一大堆关于某一主题的问题及其解决方案。我们要求学生研究每个问题及其解决方案。如果他们勤奋学习，我们期望在学生解决了一系列问题之后，他们能够找到如何从一个问题推导出答案的方法，甚至能够解决与他们所学问题相关的新问题，但这一次无需借助任何解决方案。

At the moment of its creation, our neuron cannot recognize any kind of data; it learns to recognize them. The way it learns is by example. The whole process is akin to having a student learn something by giving them a large bunch of problems on a subject, along with their solutions. We ask the student to study each problem and its solution. If they are diligent, we expect that after the student has gone through a number of problems, they will have figured out how to get from a problem to its solution and will even be able to solve new problems, related to the ones they studied, but this time without having recourse to any solutions.

当我们这样做时，我们训练计算机寻找解决方案；已解决的示例问题集称为训练数据集。这是监督学习的一个例子 因为这些解决方案会像主管一样引导计算机找到正确答案。监督学习是机器学习最常见的形式，机器学习是一门涉及训练计算机做事的方法的学科。除了监督学习之外，机器学习还包含无监督学习，我们为计算机提供训练数据集，但不提供任何附带的解决方案。无监督学习有许多重要的应用，例如将观测结果分组到不同的聚类中（对于正确的观测结果聚类没有先验解决方案）。但一般而言，监督学习比无监督学习更强大，因为我们在训练期间提供更多信息。这里我们只讨论监督学习。

When we do this, we train the computer to find the solutions; the set of solved example problems is called the training data set. This is an instance of supervised learning because the solutions guide the computer, like a supervisor, toward finding the right answers. Supervised learning is the most common form of machine learning, the entire discipline that deals with methods where we train computers to do things. Apart from supervised learning, machine learning also encompasses unsupervised learning, where we provide the computer with a training data set, but not with any accompanying solutions. There are important applications of unsupervised learning, like, for example, grouping observations into different clusters (there is no a priori solution to what a correct cluster of observations is). In general, though, supervised learning is more powerful than unsupervised learning, as we provide more information during training. We will only deal with supervised learning here.

训练结束后，学生通常会通过一些测试来检验他们对学习材料的掌握程度。同样，在机器学习中，训练结束后，我们会给计算机一个它从未见过的数据集，并让它求解这个测试数据集。然后，我们会根据机器学习系统在测试数据集中解决问题的能力来评估其性能。

After training, the student often passes some tests to see how well they mastered the material. Similarly, in machine learning, after training we give the computer another data set that it has not seen before and ask it to solve this test data set. Then we evaluate the performance of the machine learning system based on how well it manages to solve the problems in the test data set.

在分类任务中，监督学习的训练是通过给神经元网络提供大量的观察值（问题）及其对应的类别（解决方案）来实现的。我们期望神经元能够以某种方式学习如何从观察值推导出对应的类别。那么，如果我们给出对于它以前从未见过的观察结果，它应该能够合理成功地对其进行分类。

In the classification task, training for supervised learning works by giving the neuron network a large number of observations (problems) along with their classes (solutions). We expect that the neuron will somehow learn how to get from an observation to its class. Then if we give it an observation it has not seen before, it should classify it with reasonable success.

神经元对任何输入的行为都由其权重和偏差决定。开始时，我们将它们设置为随机值；神经元什么都不知道，就像一个一无所知的学生。我们以一对的形式给神经元一个输入。神经元会产生一个输出。由于权重和偏差是随机的，输出也将是随机的。然而，对于训练数据集中的每个观测值，我们都知道神经元的正确答案应该是什么。然后，我们可以计算神经元的输出与期望输出的偏差。这被称为损失：衡量神经元对给定输入的错误程度的指标。

The behavior of a neuron for any input is determined by its weights and bias. When we start, we set them at random values; the neuron knows nothing, like a clueless student. We give the neuron one input in the form of a pair. The neuron will produce an output. As we have random weights and bias, the output will also be random. For each of our observations in the training data set, however, we do know what the correct answer from the neuron should be. We can then calculate how far off the neuron’s output is from the desired one. This is called the loss: a measure of how wrong the neuron is for a given input.

例如，如果对于一个输入，神经元的输出为 0.2，而期望输出为 1.0，我们可以通过这两个值之间的差值来计算损失。为了避免符号问题，我们通常将差值的平方作为损失；这里就是。如果期望输出为 0.0，那么损失就是。无论如何，计算出损失后，我们现在可以调整权重和偏差，使其最小化。

For example, if for an input the neuron produces as output the value 0.2, while the desired output is 1.0, we can calculate the loss by the difference between the two values. To avoid having to deal with signs, we usually take as the loss the square of the difference; here it would be . If the desired output were 0.0, then the loss would be . Be it as it may, having calculated the loss, we can now adjust the weights and bias so as to minimize it.

回到人类学生的例子，每次尝试解决一个练习失败后，我们都会鼓励他们表现得更好。学生会意识到他们必须稍微改变一下方法，尝试下一个例子。如果他们如果失败了，我们就再一次提醒它们。如此反复。直到它们在训练数据集中训练了大量样本后，它们的正确率会越来越高，最终能够处理测试数据集。

Going back to the human student, after each failed attempt to solve an exercise, we nudge them to perform better. The student figures out that they have to change their approach a bit and try with the next example. If they fail, we nudge them again. And again. Until after a lot of examples in the training data set, they will start getting things right more and more, and will be able to tackle the test data set.

神经科学告诉我们，当学生学习时，大脑内部的神经连接会发生变化；神经元之间的一些突触会增强，一些会减弱，还有一些会消失。目前还没有与人工神经元直接对应的模型，但类似的事情也会发生。再次回顾一下，神经元的行为取决于其输入、权重和偏差。我们无法控制输入；它来自环境。但我们可以改变权重和偏差。这就是实际发生的情况。我们更新权重和偏差的值，以使神经元能够最小化其误差。

When a student learns, neuroscience tells us that the wiring inside the brain changes; some synapses between neurons get stronger, some get weaker, and some are dropped. There is no direct equivalent to an artificial neuron, but something similar happens. Recall once more that the behavior of a neuron depends on its input, weights, and bias. We have no control over the input; it comes from the environment. But we can change the weights and biases. And this is what really happens. We update the values of the weights and bias in such a way that the neuron will minimize its errors.

神经元实现这一点的方式是利用它所执行任务的性质。我们希望它提取每个观测值，计算出与某个类别对应的输出，并调整其权重和偏差，以最小化损失。因此，神经元试图解决一个最小化问题。给定一个输入及其产生的输出，问题是：我们如何重新校准权重和偏差，以最小化损失？

The way that the neuron achieves that is by taking advantage of the nature of the task it is called to perform. We want it to take each observation, calculate an output corresponding to a class, and adjust its weights and bias to minimize its loss. So the neuron is trying to solve a minimization problem. Given an input and the output it produces, the problem is, How are we to recalibrate the weights and bias to minimize the loss?

这需要我们转变概念。到目前为止，我们将神经元描述为接受输入并产生输出的实体。从这个角度来看，整个神经元是一个接受输入的大型函数，应用权重，对乘积求和，加上偏差，将结果传入激活函数，最终得到最终输出。但如果我们换个角度思考，我们的输入和输出实际上是给定的（也就是我们的训练数据集），而我们能改变的是权重和偏差。因此，我们可以将整个神经元视为一个函数，其变量是权重和偏差，因为这些才是我们真正能够影响的，并且对于每个输入，我们都希望改变它们，以最小化损失。

This requires a conceptual change of focus. Up to this point we have described a neuron as something that takes some inputs and produces an output. Viewed in this way, the whole neuron is a big function that takes its inputs, applies the weights, sums the products, adds the bias, passes the result through the activation function, and produces the final output. But if we think of it another way, our inputs and outputs are actually given (that is our training data set), while what we can change are the weights and bias. So we can view the whole neuron as a function whose variables are the weights and bias because these are what we can really affect, and for every input we want to change them so as to minimize the loss.

如果我们以一个简单的神经元为例，它只有一个权重并且没有偏差，那么损失和权重之间的关系可能如下一页图片的左侧所示。粗曲线表示对于给定输入，损失与权重的关系。神经元应该调整其权重，以使其达到函数的最小值。对于给定的输入，神经元目前在指示点有损失。不幸的是，神经元不知道可以使损失最小化的理想权重是多少，因为它唯一知道的就是函数在指示点的值；它并不像我们在图中那样拥有有利的视角。神经元只能小幅调整其权重（增加或减少），以使其更接近最小值。

If we take as an illustration a simple neuron, with just one weight and no bias, then the relationship between the loss and weight might be as in the left part of the figure on the next page. The thick curve shows the loss as a function of the weight for a given input. The neuron should adjust its weight so that it reaches the minimum value of the function. The neuron, for the given input, has currently a loss at the indicated point. Unfortunately, the neuron does not know what is the ideal weight that would minimize the loss, given that the only thing it does know is the value of the function at the indicated point; it is not endowed with a vantage point of view like we have with the figure at our disposal. The neuron may only adjust its weight by a small amount—either increase or decrease it—so that it moves closer to the minimum.

为了确定该做什么，是增加还是减少权重，神经元可以找到当前点的切线。然后它可以计算切线的斜率线；这是与水平轴的夹角，我们也在图中显示了出来。注意，神经元除了能够在局部点进行计算之外，不需要任何特殊能力就能做到这一点。由于角度是顺时针的，所以切线的斜率为负。斜率表示函数的变化率；因此，负斜率表示通过增加权重，损失会减少。神经元由此发现，要减少损失，它必须向右移动。由于斜率为负，而所需的权重变化为正，神经元发现它必须将权重移到正方向——与斜率指示的方向相反。

To find out what to do, whether to increase or decrease the weight, the neuron can find the tangent line at the current point. Then it can calculate the slope of the tangent line; this is the angle with the horizontal axis, which we have also shown in the figure. Note that the neuron can do that without any special capabilities apart from being able to carry out calculations at the local point. The slope of the tangent is negative because the angle is clockwise. The slope shows the rate of change of a function; therefore a negative slope indicates that by increasing the weight, the loss decreases. The neuron thereby discovers that to decrease the loss, it has to move to the right. As the slope is negative and the required change in the weight is positive, the neuron finds that it must move the weight in a positive direction—opposite to what is indicated by the slope.

现在翻到右图。这次神经元位于最小损失函数的右侧。它再次取正切值并计算其斜率。因此，斜率为正。正斜率表示增加权重会导致损失增加。神经元知道，为了最小化损失，必须降低权重。由于斜率为正，而所需的权重变化为负，神经元再次发现它必须朝与斜率指示方向相反的方向移动。

Now turn to the figure on the right. This time the neuron is to the right of the minimum loss. It takes the tangent again and calculates its slope. The angle and therefore slope is positive. A positive slope indicates that by increasing the weight, the loss increases. The neuron then knows that in order to minimize the loss, it has to decrease the weight. As the slope is positive and the required change in the weight is negative, the neuron finds again that it must move in the opposite direction than that indicated by the slope.

因此，在这两种情况下，规则都是相同的：神经元计算斜率，并沿与斜率相反的方向更新权重。这些在微积分中可能看起来很熟悉。函数在某一点的斜率就是它的导数。为了减少损失，我们需要对权重进行一个与损失导数相反的小幅调整。

In both cases, then, the rule is the same: the neuron calculates the slope and updates the weight in the opposite direction from the slope. All this might look familiar from calculus. The slope of a function at a point is its derivative. To decrease the loss, we need to change the weight by a small amount that is opposite to the derivative of the loss.

现在，神经元通常不只有一个权重，而是有多个权重，并且还有一个偏差。为了找到如何调整每个权重和偏差的方法，神经元会像我们之前描述的单个权重那样进行操作。用数学术语来说，它会计算损失函数对每个权重和偏差的偏导数。对于n 个权重和一个偏差，它们总共是偏导数。包含函数所有偏导数的向量称为其梯度。梯度相当于多变量函数的斜率；它表示我们必须沿着哪个方向移动才能增加函数的值。要减少函数值，则朝相反的方向移动。因此，为了减少损失，神经元会更新每个权重和偏差。与形成梯度的偏导数所指示的方向相反。5

Now a neuron does not usually have a single weight but rather has many, and also has a bias. To find out how to adjust each individual weight and the bias, the neuron proceeds like we described for the single weight. In mathematical terms, it calculates the so-called partial derivative of the loss with respect to each individual weight and bias. For n weights and a bias, that will be partial derivatives in total. A vector containing all the partial derivatives of a function is called its gradient. The gradient is the equivalent of the slope when we have multivariable functions; it shows the direction along which we have to move to increase the value of the function. To decrease it, we move in the opposite direction. Thus to decrease the loss, the neuron updates each weight and the bias in the opposite direction than the one indicated by the partial derivatives forming its gradient.5

计算并非真正通过绘制切线和测量角度来完成。有一些有效的方法可以找到偏导数和梯度，但我们无需深入探讨细节。重要的是，我们有一个明确定义的方法来调整权重和偏差，以改善神经元的结果。有了这个，学习过程可以用以下算法来描述：

The calculations are not really performed by drawing tangents and measuring angles. There are efficient ways to find the partial derivatives and gradient, but we don’t need to get into the details. What is important is that we have a well-defined way to adjust the weights and bias to improve the results of the neuron. With this at hand, the learning process can be described by the following algorithm:

• 对于训练数据集中的每个输入和期望输出，
  1. 1.计算神经元的输出和损失。
  2. 2. 更新神经元的权重和偏差，以最小化损失。
• For each input and desired output in the training data set,
  1. 1. Calculate the output of the neuron and loss.
  2. 2. Update the weights and bias of the neuron to minimize the loss.

一旦我们通过遍历训练数据集中的所有数据完成了一次训练，我们就说我们已经完成了一个epoch。通常我们不会就此止步。我们会重复整个过程若干个 epoch；就好像学生在学习了所有学习材料后，又从头开始。我们期望他们下一次会做得更好，因为这一次他们不是从零开始——他们并非完全无知——他们已经从上一个 epoch 中学到了一些知识。

Once we have completed a training by going through all the data in the training data set, we say that we have completed an epoch. Usually we do not leave it at this. We repeat the whole process for a number of epochs; it is as if the student, after going through all the study material, started all over again. We expect that the next time they’ll do better, as this time they do not start from zero—they are not completely clueless—having already learned something from the previous epoch.

我们通过在训练方案中添加时期来重复训练的次数越多，我们对训练数据的掌握就越好。但训练过多也可能有害。反复学习同一组问题的学生可能会死记硬背地学习解决问题，而并不真正知道如何解决以前没有遇到过的任何其他问题。我们经常看到这种情况，一个看似准备充分的学生在考试中惨败。在机器学习中，当我们在训练数据集上训练计算机时，我们说它与数据拟合。过多的训练会导致所谓的过度拟合：训练数据集表现优异，而测试数据集表现不佳。

The more we repeat the training by adding epochs into our training regime, the better we get with the training data. But too much training can be a bad thing. A student who studies again and again the same set of problems will probably learn to solve them by rote—without really knowing how to solve any other problems that they have not encountered before. We see that happening when a seemingly well-prepared student fails abysmally in the exams. In machine learning, when we train the computer on a training data set, we say that it fits the data. Too much training results in what is called overfitting: excellent performance with the training data set, and bad performance with the test data set.

可以证明，按照该算法，神经元可以学习对任何线性可分的数据进行分类。如果我们的数据是二维的（就像我们的例子一样），那么这意味着它们应该可以通过一条直线分离。如果我们的数据具有更多特征，而不仅仅是，则该原理具有普遍性。对于三维（即三个输入），如果数据可以通过三维空间中的一个简单的平面分离，则它们就是线性可分的。对于更多维度，我们将线和平面的等价物称为超平面。

It can be proven that following this algorithm, a neuron can learn to classify any data that are linearly separable. If our data have two dimensions (like our example), then that means that they should be separable by a straight line. If our data have more features, not just , the principle is generalized. For three dimensions—that is, three inputs —the data are linearly separable if they can be separated by a simple plane in the three-dimensional space. For more dimensions, we call the equivalent of the line and plane a hyperplane.

训练结束时，我们的神经元已经学会了如何分离数据。“学会”意味着它找到了正确的权重和偏差，就像我们描述的那样：它从随机值开始，然后逐渐更新它们。最小化损失。回想一下图中的两个斑点，神经元学会了用决策边界来区分它们。我们从左下方的神经元，到右边的神经元，你可以看到它的参数的最终值。

At the end of the training, our neuron has learned to separate the data. “Learned” means that it has found the right weights and bias, in the way we described: it started out with random values and then gradually updated them, minimizing the loss. Recall the figure with the two blobs, which the neuron learned to separate with a decision boundary. We got from the neuron below at the left, to the neuron at the right, where you can see the final values of its parameters.

这并非总是如此。单个神经元单独运作时只能执行某些任务，例如这种线性可分数据的分类。为了处理更复杂的任务，我们需要从单个人工神经元转向神经元网络。

That does not always happen. A single neuron, acting alone, can only perform certain tasks, like this classification of linearly separable data. To handle more complicated tasks, we need to move from a lone artificial neuron to networks of neurons.

从神经元到神经网络

From Neurons to Neural Networks

就像生物神经网络一样，我们可以用相互连接的神经元构建人工神经网络。一个神经元的输入信号可以连接到其他神经元的输出，而它的输出信号也可以连接到其他神经元的输入。通过这种方式，我们可以创建如下神经网络：

As in biological neural networks, we can build artificial neural networks out of interconnected neurons. The input signals of a neuron can be connected to the outputs of other neurons, and its output signal can be connected to the inputs of other neurons. In this way we can create neural networks like this one:

这个人工神经网络的神经元是分层排列的。这在实际应用中很常见：我们构建的许多神经网络都是由多层神经元构成的，每一层都紧挨着前一层。我们还让一层的所有神经元都与下一层的所有神经元连接，从左到右。这虽然不是必需的，但也很常见。当我们将层以这种方式连接时，我们称之为“密集连接”。

This artificial neural network has its neurons arranged in layers. This is often done in practice: many neural networks that we construct are made of layers of neurons, with each layer stacked next to a previous one. We have also made all the neurons on one layer connect to all the neurons on the next layer, going from left to right. This, again, is common, although not necessary. When we have layers connected like that, we call them densely connected.

虽然第一层与前一层没有连接，但最后一层的输出同样也不与后一层连接。最后一层的输出是整个网络的输出；它将提供我们想要计算的值。

While the first layer is not connected to any previous one, the output of the last layer is similarly not connected to any following layer. The output of the last layer is the output of the whole network; it will provide the values that we want it to calculate.

让我们回到分类任务。我们现在的问题是区分两组数据，如下页上图所示。这些数据分布在同心圆中。人类很容易就能看出它们属于两个不同的组。同样明显的是，它们不是线性可分的：没有直线可以区分这两类。我们希望创建一个能够区分这两组的神经网络，以便它能够告诉我们未来的任何观察结果将属于哪个组。这就是你在下图中看到的。对于任何在浅色背景下的观察结果，神经网络会识别出它属于一个组；对于任何在深色背景下的观察结果，它会告诉我们它属于另一个组。

Let us return to a classification task. Our problem now is to pick apart two sets of data, shown in the figure on the top of the next page. The data fall into concentric circles. It is clear to a human that they belong to two distinct groups. It is also clear that they are not linearly separable: no straight line can separate the two classes. We want to create a neural network that will be able to tell the two groups apart so that it will tell us in which group any future observation will belong. This is what you see in the figure at the bottom. For any observation on the light background, the neural network will recognize that it belongs to one group; for any observation on the dark background, it will tell us that it belongs to the other group.

为了达到下图所示的结果，我们逐层构建了一个网络。我们在输入层放置两个神经元，每个神经元对应一个数据坐标。我们添加一个包含四个神经元的层，并与输入层紧密连接。由于该层不与输入或输出连接，因此它是一个隐藏层。我们添加另一个包含两个神经元的隐藏层，并与第一个隐藏层紧密连接。最后，我们用一个包含一个神经元的输出层来完成网络，该输出层与最后一个隐藏层紧密连接。所有神经元使用 tanh 激活函数。输出神经元将产生一个介于和之间的值，表明它认为数据属于其中一类。我们将根据该值是否超过 0.0，将其转换为二元决策，即“是”或“否”。神经网络如下所示：

To achieve the results that we see in the lower figure, we build a network layer by layer. We put two neurons on the input layer, one for each coordinate of our data. We add one layer with four neurons, densely connected to the input layer. Because this layer is not connected to the input or output, it is a hidden layer. We add another hidden layer with two neurons, densely connected to the first hidden layer. We finish the network with an output layer of one neuron, densely connected to the last hidden layer. All the neurons use the tanh activation function. The output neuron will produce a value between and , displaying its belief that the data fall in one or the other group. We’ll take that value and turn it into a binary decision, yes or no, depending on whether it exceeds 0.0 or not. This is what the neural network looks like:

反向传播算法

The Backpropagation Algorithm

一开始，神经网络一无所知，也没有进行任何调整；我们从随机的权重和偏差开始。这就是神经网络世界中“无知”的含义。然后，我们给神经网络一个来自数据的观测值，也就是一组坐标。和坐标将放在输入层。两个神经元都接受和值，并将它们作为输出传递到第一个隐藏层。所有四个神经元该层计算其输出，并将其发送到第二个隐藏层。该层的神经元将其自身的输出发送到输出层的神经元，从而产生神经网络的最终输出值。随着计算逐层进行，神经网络将神经元的结果从输入层向前传播到输出层：

In the beginning, the neural network knows nothing, and no adjustment has taken place; we start with random weights and biases. This is what ignorance means in the neural network world. Then we give the neural network an observation from our data—that is, a set of coordinates. The and coordinates will go on the input layer. Both neurons take the and values and they pass them as their output to the first hidden layer. All four neurons of that layer calculate their output, which in their turn, they send to the second hidden layer. The neurons on that layer send their own output to the neuron on the output layer, which produces the final output value of the neural network. As the calculations proceed from layer to layer, the neural network propagates the results of the neurons forward, from the input to the output layer:

一旦到达输出层，我们就会像计算单个神经元一样计算损失。然后，我们不仅要调整单个神经元的权重和偏差，还要调整网络中的所有神经元，以最小化损失。

Once we reach the output layer, we calculate the loss, as we did with the single neuron. And then we want to adjust the weights and bias of not just one neuron but rather all the neurons in the network so as to minimize the loss.

事实证明，从输出层到输入层，反过来也可以做到这一点。一旦我们知道了损失，我们就可以更新权重，并且输出层神经元的偏差（这里我们只有一个神经元，但并非总是如此）。更新完输出层的神经元后，我们可以更新其前一层（最后一个隐藏层）的神经元的权重和偏差。完成之后，我们可以更新其前一层（倒数第二个隐藏层）的权重和偏差。依此类推，直到到达输入层：

It turns out that it is possible to do that by going in the opposite direction, from the output to the input layer. Once we know the loss, we can update the weights and biases of the neurons on the output layer (here we have just a single neuron, but this need is not always so). Having updated the neurons on the output layer, we can update the weights and biases of the neurons on the layer before that—the last hidden layer. Having done that, we can update the weights and biases of the layer before that—the one-but-last hidden layer. And so on, until we reach the input layer:

神经元权重和偏差的更新方式与单个神经元的更新方式类似。同样，更新是基于数学导数计算的。你可以将整个神经网络想象成一个巨大的函数，其变量是所有神经元的权重和偏差。然后，我们可以计算每个权重和偏差对损失的导数，并使用该导数来更新神经元。由此，我们了解了神经网络学习过程的核心：反向传播算法。6

The way the weights and biases of the neurons are updated is similar to the way a single neuron is updated. Again, the updates are calculated based on mathematical derivatives. You can think of the whole neural network as an enormous function whose variables are the weights and biases of all the neurons. Then we can calculate the derivative of each and every weight and bias with respect to the loss, and use that derivative to update the neuron. With this we arrive at the heart of the learning process in neural networks: the backpropagation algorithm.6

• 对于每个输入和期望输出，
  1. 1. 从输入层到输出层，逐层计算神经网络的输出和损失。
  2. 2. 更新神经元的权重和偏差，以最小化损失，从输出层反向到输入层。
• For each input and desired output,
  1. 1. Calculate the output and loss of the neural network proceeding layer by layer, going forward from the input to the output layer.
  2. 2. Update the weights and biases of the neurons to minimize the loss, going backward from the output to the input layer.

利用反向传播算法，我们可以构建复杂的神经网络，并训练它们执行不同的任务。深度学习系统的构建模块很简单。它们是人工神经元，计算能力有限：接受输入、乘以权重、求和、添加偏差，并对结果值应用激活函数。它们的能力源于以特殊方式连接大量神经元，由此产生的网络可以被训练来执行我们期望它们执行的任务。

Using the backpropagation algorithm, we can build complex neural networks and train them to perform different tasks. The building blocks of deep learning systems are simple. They are artificial neurons, with their limited computational capabilities: taking inputs, multiplying by weights, summing, adding a bias, and applying an activation function on the resulting value. Their power derives from connecting lots and lots of them in special ways, where the resulting networks can be trained to perform the task that we want them to perform.

识别衣服

Recognizing Clothes

为了使讨论更加具体，我们假设我们想要建立一个识别物品的神经网络图像中显示的服装，所以这将是一个图像识别任务。神经网络在这方面表现得非常出色。

To render the discussion more concrete, let us assume that we want to build a neural network that recognizes items of clothing displayed in images, so this is going to be an image recognition task. Neural networks have been found to be exceptionally good at this.

每幅图像都是一张小照片，尺寸为。我们的训练数据集包含 60,000 幅图像，测试数据集包含 10,000 幅图像；我们将使用其中 60,000 幅图像来训练神经网络，另 10,000 幅图像用于评估其学习效果。以下是一张示例图像，我们为其添加了坐标轴和网格，以便于后续讨论：7

Each image will be a small photo, of dimensions . Our training data set consists of 60,000 images, and our test data set consists of 10,000 images; we’ll use 60,000 images for training the neural network, and another 10,000 images for evaluating how well it learned. Here is an example image, on which we have added axes and a grid to help the discussion that follows:7

图像被分解成不同的小块，因为这是我们处理数字图像的方式。我们将整幅图像视为一个矩形区域，并将其分成许多小块，每个小块被赋予一个从 0 到 255 的整数值，对应一种灰度，其中 0 表示全白，255 表示全黑。上图实际上就是下一页的矩阵。

The image is broken into small distinct parts because that is how we handle images digitally. Taking the whole image as a rectangular plot, we divide it into small patches, of them, and each patch is given an integer value from 0 to 255, corresponding to a shade of gray, with 0 being completely white and 255 being completely black. The above image is actually the matrix on the following page.

实际上，神经网络通常要求我们将其输入缩放到一个较小的值域，例如 0 到 1 之间，否则它们可能无法正常工作；你可能会认为较大的输入值会导致神经元误入歧途。这意味着在使用此矩阵之前，我们需要将每个单元除以 255，但在后续讨论中我们将忽略这一点。

In reality, neural networks require that we usually scale their inputs to a small range of values, such as between 0 and 1, otherwise they may not work well; you may think of it as having large input values that lead neurons astray. That means that before using this matrix we would divide each cell by 255, but we’ll ignore this in the rest of the discussion.

不同的服装可能属于十个不同的类别，如下表所示。对于计算机来说，类别只是不同的数字，我们称之为标签：

The different items of clothing may belong to ten different classes, which you can see in the table below. To a computer, the classes are just different numbers, which we call labels:

下图展示了每种服装的随机样本，共十件。正如您所见，这些图片种类繁多，并非所有图片都能完美地展现每个特定服装类别。这使得问题变得更加有趣。我们想要创建一个神经网络，将类似这样的图片作为输入，并输出一个输出，告诉我们它认为输入属于哪种类型的图片。

In the following figure, we show a random sample of ten items from each kind of clothing. There is quite a variety in the images, as you can see, and not all of them are picture-perfect examples of each particular clothing class. That makes the problem somewhat more interesting. We want to create a neural network that takes as its input images like these and provides an output that tells us what kind of image it believes its input is.

同样，我们将分层构建神经网络。第一层包含输入神经元，共有 784 个神经元。每个神经元都从图像中的单个块中获取单个输入，并输出其输入的值。如果图像是踝靴，则第一个神经元的输入将为左上角块中的值 0，并将该值输出。其余神经元将按行顺序（从上到下、从左到右）获取块的值。靴子后跟右端（倒数第四行、右数第三列）值为 58 的块将获取该值 58 并将其复制到输出。由于神经网络的行列数是从上到左计算的，因此该神经元位于从上到下第 25 行、从左到右第 26 列，因此它就是输入神经元编号。

Again, we’ll build our neural network in layers. The first layer, comprising the input neurons, will have 784 neurons. Each one of them will take a single input, from a single patch in the image, and will simply output the value that it gets in its input. If the image is the ankle boot, the first neuron will get the value in the top-left patch, a 0, in its input, and it will output that 0. The rest of the neurons will get the values of the patches proceeding row wise, from top to bottom, left to right. The patch with the value 58, at the right end of the heel of the boot (the fourth row from the bottom, and the third column from the right) will get this 58 and copy it on its output. As rows and columns are counted in the neural network from the top and left, this neuron is in the twenty-fifth row from the top and twenty-sixth column from the left, making it the input neuron number .

下一层将与输入层紧密连接。它由 128 个 ReLU 神经元组成。该层不直接连接到输入图像（输入层直接连接），也不会直接连接到输出（我们将为此添加另一个层）。因此，它是一个隐藏层，因为我们无法从神经网络外部观察到它。由于是紧密连接的，这将导致输入层和隐藏层之间有大量的连接。隐藏层上的每个神经元都将连接到输入层上所有神经元的输出。每个神经元有 784 个输入连接，总共有连接数。

The next layer will be densely connected to the input layer. It will consist of 128 ReLU neurons. This layer is not directly connected to the input images (the input layer is) and will not be directly connected to the output (we’ll add another layer for that). Therefore it is a hidden layer, as we cannot observe it from the outside of the neural network. Being densely connected, this will result in a large number of connections between the input and hidden layer. Each neuron on the hidden layer will be connected to the outputs of all neurons on the input layer. There will be 784 input connections per neuron, for a total of connections.

我们将添加另一个最后一层，它将包含输出神经元，用于传递神经网络的结果。它将包含 10 个神经元，每个类别一个。每个输出神经元将连接到隐藏层的所有神经元，总共连接数为 个。神经网络中所有层之间的连接总数为。最终的神经网络工作以示意图的形式呈现，类似于下一页的示意图。由于不可能将所有节点和边都放进去，您可以看到虚线框代表输入层和隐藏层上的大部分节点；第一个框中有 780 个节点，第二个框中有 124 个节点。我们还折叠了指向框内各个节点的箭头。

We will add another, last layer, which will contain the output neurons that will carry the results of the neural network. This will contain 10 neurons, one for each class. Each output neuron will be connected to all the neurons of the hidden layer, for a total of connections. The grand total of all the connections between all the layers in the neural network will be . The resulting neural work will look, in schematic form, like the one on the next page. As it is impossible to fit all the nodes and edges, you can see dotted boxes standing for the bulk of nodes on the input and hidden layers; there are 780 nodes in the first box and 124 nodes in the second box. We have also collapsed the arrows going to the individual nodes inside the boxes.

我们的神经网络的输出将由 10 个输出组成，分别来自该层上的每个神经元。每个输出神经元代表一个类，其输出表示输入图像属于该类的概率；所有 10 个神经元的概率之和为 1，正如我们处理概率时必然发生的那样。这是另一个激活函数的例子，称为softmax，它将实数向量作为输入，并将其转换为概率分布。让我们看接下来的两个例子。

The output of our neural network will consist of 10 outputs, one from each neuron on the layer. Each output neuron will represent one class, and its output will represent the probability that the input image belongs to this class; the sum of the probabilities of all 10 neurons will be 1, as it must happen when we deal with probabilities. This is an example of yet another activation function, called softmax, which takes as input a vector of real numbers and converts them to a probability distribution. Let’s see the two examples that follow.

在左边的第一个例子中，经过训练后我们在网络的输出中得到了这个：

In the first example, on the left, after training we get this at the output of the network:

这意味着神经网络告诉我们，它非常肯定它正在处理一件连衣裙，其概率为 83%，而输入图像是 T 恤/上衣、衬衫或裤子的概率则很小。

That means that the neural network tells us that it is pretty certain it is dealing with a dress, giving it an 83 percent probability, leaving aside small probabilities for the input image being a T-shirt/top, shirt, or trouser.

在右侧的第二个示例中，网络产生：

In the second example, on the right, the network produces:

神经网络有 43% 的把握认为它正在处理一件衬衫——但它错了；这张照片实际上是一件套头衫（以防你看不出来）。不过，它还是给出了第二高的准确率，33%，认为这张照片是一件套头衫。

The neural network is 43 percent certain that it is dealing with a shirt—and it is wrong; the photo is really a picture of a pullover (in case you couldn’t tell). Still, it did give its second best, at 33 percent, to the image being a pullover.

我们给出了一个网络得出正确答案的例子，以及另一个网络得出了错误的答案。总的来说，如果我们给网络输入大量图像进行识别，包括训练数据集中的所有 60,000 幅图像，我们会发现它在测试数据集的 10,000 幅图像中大约能识别出 86% 的正确率。考虑到这个神经网络虽然比之前的复杂得多，但仍然很简单，这个结果还不错。基于这个基础，我们可以创建更复杂的网络结构，从而获得更好的结果。

We gave one example where the network comes up with the right answer, and another instance where the network comes up with the wrong answer. Overall, if we give the network many images to recognize, all the 60,000 images in our training data set, we’ll find out that it manages to get right about 86 percent of the 10,000 images in the test data set. That is not bad, considering that the neural network, even though it is way more complicated than the previous one, is still a simple one. From this baseline, we can create more complicated network structures that would give us better results.

尽管复杂度有所提升，我们的神经网络的学习方式与识别数据块和同心圆的简单网络相同。训练期间，每个输入都会获得一个输出，我们会将其与期望输出进行比较，从而计算损失。现在的输出不再是一个单一值，而是 10 个值，但原理相同。当神经网络以约 83% 的概率识别一件衬衫时，我们可以将其与理想状态（即 100% 的概率）进行比较。因此，我们有两组输出值：一组是由网络获得的，不同种类的衣服被赋予不同的概率；另一组是我们希望从网络获得的概率，即一组概率，除了一个对应于正确答案的概率（等于 1）外，其他所有概率都为零。在最后一个示例中，与目标值对比的输出如下：

Despite the increased complexity, our neural network learns in the same way as our simpler networks recognizing blobs of data and concentric circles. For each input during training we obtain an output, which we compare to the desired output to calculate the loss. The output now is not a single value but rather 10 values, yet the principle is the same. When the neural network recognizes a shirt with about 83 percent probability, we can compare that with the ideal, which would be to recognize it with 100 percent probability. Therefore we have two sets of output values: the one obtained by the network, with various probabilities assigned to the different kinds of clothes, and what we would like to have gotten from the network, which is a set of probabilities where all of them are zero apart from a single probability, corresponding to the right answer, which is equal to one. In the last example, the output contrasted to the target would be as follows:

我们取最后两列，再次计算损失指标——只是这一次，由于我们没有单个值，所以我们不计算简单的平方差。存在一些指标可以计算此类值集之间的差异。在我们的神经网络中，我们使用了一种称为分类交叉熵的指标，它表示两个概率分布的差异程度。计算损失后，我们更新输出层的神经元。更新后，我们更新隐藏层的神经元。简而言之，我们进行反向传播。

We take the last two columns and we calculate again a loss metric—only this time, as we do not have a single value, we do not calculate a simple squared difference. There exist metrics to calculate the difference between sets of values like these. In our neural network we used one such metric, called categorical cross-entropy, which indicates how much two probability distributions differ. Having calculated the loss, we update the neurons on the output layer. Having updated them, we update the neurons on the hidden layer. In short, we perform backpropagation.

我们对训练数据集中的所有图像（也就是整个epoch）都进行相同的处理。完成后，我们会再重复这个过程，进行下一个epoch。我们重复这个过程，并努力达到一个平衡：足够训练周期数 (epoch) 是为了使神经网络能够从训练数据集中尽可能多地学习，而不会因为训练周期数过多而导致神经网络从训练数据集中学习过多。在学习过程中，神经网络会调整其神经元的权重和偏差，这些神经元的数量非常多。输入层只是将值复制到隐藏层，因此无需对输入神经元进行任何调整。但是，隐藏层有 100,352 个权重，输出层有 1,280 个权重，隐藏层有 128 个偏差，输出层有 10 个偏差，总共有 101,770 个参数。

We go through the same process for all images in our training data set—that is, for a whole epoch. When we are done, we do this all over again for another epoch. We repeat the process while trying to strike a balance: enough epochs so that the neural network will learn as much as possible from the training data set without going into too many epochs where the neural network will learn too much from the training data set. During learning, the network will be adjusting the weights and biases of its neurons, which are a lot. The input layer just copies values to the hidden layer, so no adjustments need to be done to the input neurons, but there are 100,352 weights on the hidden layer, 1,280 weights on the output layer, 128 biases on the hidden layer, and 10 biases on the output layer, for a total of 101,770 parameters.

深入学习

Getting to Deep Learning

可以证明的是，尽管神经元本身功能有限，但神经网络可以执行任何能够用算法描述并在计算机上运行的计算任务。因此，计算机能做的，神经网络都能做到。当然，其核心思想是，我们不需要告诉神经网络如何执行任务。我们只需要在使用算法的同时，向它输入样本，让神经网络学习如何执行任务。我们已经看到，反向传播就是这样一种算法。虽然我们将样本限制在分类任务上，但神经网络可以应用于各种不同的任务。它们可以预测神经网络能够识别目标量（例如信用评分）的值，进行语言间翻译，以及理解和生成语音；并在围棋比赛中击败人类冠军，并在此过程中展示了这项古老游戏的全新策略，令专家们感到困惑。它们甚至学会了如何在不了解规则的情况下下围棋，无需访问先前对局的数据库，然后像神经网络在与自己对弈一样进行学习。8

It can be proven that even though a neuron on its own cannot do much, a neural network can perform any computational task that can be described algorithmically and run on a computer. Therefore there is nothing that a computer can do that a neural network could not do. The whole idea, of course, is that we do not need to tell the neural network exactly how to perform a task. We only need to feed it with examples while using an algorithm to make the neural network learn how to perform the task. We saw that backpropagation is such an algorithm. Although we limited our examples to classification, neural networks can be applied to all sorts of different tasks. They can predict the values of a target quantity (for instance, credit scoring), translate between languages as well as understand and generate speech; and beat human champions in the game of Go, in the process baffling experts by demonstrating completely new strategies of playing a centuries-old game. They have even learned how to play the game of Go starting with just a knowledge of the rules, without access to a library of previously played games, and then proceeding to learn as if the neural network were playing games against itself.8

如今，神经网络的成功应用比比皆是，但其原理却并非新鲜事物。感知器发明于 20 世纪 50 年代，反向传播算法也已有 30 多年的历史。在此期间，神经网络的兴起和衰落，人们对其潜力的热情也随之消退。过去几年真正发生改变的是我们构建真正大型神经网络的能力。这得益于制造专用计算机芯片的进步，这些芯片可以高效地执行神经元执行的计算。如果将神经网络的所有神经元想象成排列在计算机内存中的阵列，那么所有必要的计算都可以通过对庞大的数字矩阵进行运算来完成。神经元计算其输入加权乘积的和；如果你还记得上一章关于 PageRank 的讨论，乘积的和正是矩阵乘法的本质。

Today, successful applications of neural networks abound, yet the principles are not new. The Perceptron was invented in the 1950s, and the backpropagation algorithm is more than 30 years old. In this period, neural networks came and went out of fashion, with enthusiasm for their potential ebbing and flowing. What has really changed in the last few years is our capability to build really big neural networks. This has been achieved thanks to the advances in manufacturing specialized computer chips that can perform the calculations executed by neurons efficiently. If you picture all the neurons of a neural network arranged inside a computer’s memory, then all the required computations can be carried out by operations on vast matrices of numbers. A neuron calculates the sums of the weighted products of its inputs; if you recall the discussion on PageRank in the previous chapter, the sum of the products is the essence of matrix multiplication.

事实证明，图形处理单元( GPU ) 非常适合此目的。GPU 是专门设计用于在计算机内部创建和处理图像的计算机芯片；该术语建立在中央处理器( CPU ) 的基础上，CPU 是执行计算机内部程序指令的芯片。GPU 旨在执行计算机图形指令。计算机图形的生成和处理需要对大型矩阵进行数值运算；计算机生成的场景就是一个巨大的数字矩阵（想一想鞋子）。GPU 是游戏机的主力。这种在数小时的娱乐中让人类智力停滞的技术也用于提升机器智能。

It has turned out that graphics processing units (GPUs) are perfectly suited for this. GPUs are computer chips that are specially designed to create and manipulate images inside a computer; the term builds on central processing units (CPUs), the chip that carries out the instructions of a program inside a computer. GPUs are built to carry out instructions for computer graphics. The generation and processing of computer graphics requires numerical operations on big matrices; a computer-generated scene is a big matrix of numbers (think of the shoe). GPUs are the workhorses of game consoles. The same technology that arrests human intelligence in hours of diversion is also used to advance machine intelligence.

我们从最简单的神经网络开始，它由一个神经元组成。然后我们添加了几个神经元，之后又添加了几百个神经元。尽管如此，我们创建的图像识别神经网络规模并不大，其架构也并不复杂。我们只是一层一层地添加了神经元。深度学习领域的研究人员在神经网络设计方面取得了长足的进步。这些架构可能包含数十层。这些层的几何形状不必像我们这里展示的那样，只是简单的一维神经元集合。例如，层内的神经元可以堆叠在二维画布状结构上。此外，不必将每一层都与前一层紧密连接；其他连接模式包括不可能。也不必将某一层的输出简单地连接到下一层的输入。例如，我们可以在非连续层之间建立连接。我们可以将各层捆绑在一起，将其视为模块，并将它们与包含其他层的模块组合，以形成越来越复杂的配置。如今，我们拥有种类繁多的神经网络架构，因此某些架构可以很好地适用于特定的任务。

We started with the simplest possible neural network, consisting of a single neuron. Then we added a few neurons, and then we added a few more hundreds. Still, the image recognition neural network that we created is by no means a big one. Nor is its architecture complicated. We just added layer on layer of neurons. Researchers in the field of deep learning have made big strides in devising neural network designs. These architectures may comprise dozens of layers. The geometry of these layers need not be a simple one-dimensional set of neurons, like the ones we have here. For example, neurons inside a layer may be stacked on two-dimensional canvas-like structures. Moreover, it is not necessary to have each layer densely connected to the one before; other connection patterns are possible. Nor is it necessary to have the outputs of a layer simply connected to the inputs of the next layer. We may, for instance, have connections between non-consecutive layers. We may bundle up layers and treat them as modules, combining them with modules containing other layers to form more and more complex configurations. Today we have a menagerie of neural network architectures at our disposal, such that particular architectures are well suited for specific tasks.

所有神经网络架构中，层上的神经元都会在学习过程中更新权重和偏差的值。如果我们反思一下正在发生的事情，就会发现，我们有一组输入，它们在学习过程中对层进行变换。训练停止后，这些层会通过调整参数，以某种方式吸收输入数据所代表的信息。层的权重和偏差配置代表了它接收到的输入。第一个隐藏层与输入层直接接触，对神经网络的输入进行编码。第二个隐藏层对与其直接连接的第一个隐藏层的输出进行编码。随着我们逐渐深入多层网络，每一层都会对前一层接收到的输出进行编码。每个表示都建立在前一个表示的基础上，因此比前一层的抽象程度更高。因此，深度神经网络会学习一个概念的层次结构，不断迈向越来越高的抽象层次。我们所说的深度学习正是在这个意义上。我们指的是一种架构，其中连续的层次代表着更深层次的概念，对应于更高的抽象层次。在图像识别中，多层网络的第一层可以学习识别小的局部模式，例如图像中的边缘。然后，第二层可以学习识别由第一层识别出的模式构建的模式，例如眼睛、鼻子和耳朵。第三层可以学习识别由第二层识别出的模式构建的模式，例如人脸。现在您可以看到，我们用于识别图像的神经网络有些幼稚；我们并没有尝试实现真正的深度学习。通过在抽象之上构建抽象，我们期望我们的网络能够找到人类能够发现的模式，从句子结构到医学图像中的恶性肿瘤迹象，再到识别手写字符，再到检测网络欺诈。

The neurons on the layers in all the neural network architectures update the values of the weights and biases as they learn. If we reflect on what is happening, we can see that we have a set of inputs that transforms the layers during the learning process. Once the training stops, the layers have somehow, via the adjustments in their parameters, taken in the information represented by the input data. The weights and biases configuration of a layer represents the input it has received. The first hidden layer, which comes in direct contact with the input layer, encodes the neural network’s input. The second hidden layer encodes the output of the first hidden layer, to which it is directly connected. As we proceed deeper and deeper into a multilayer network, each layer encodes the output received by the previous layer. Each representation builds on the previous one and therefore is on a higher level of abstraction from the one of the preceding layer. Deep neural networks, then, learn a hierarchy of concepts, proceeding to higher and higher levels of abstraction. It is in this sense that we talk of deep learning. We mean an architecture whereby successive levels represent deeper concepts, corresponding to higher levels of abstraction. In image recognition, the first layer of a multilayer network may learn to recognize small local patterns, such as edges in the image. Then the second layer may learn to recognize patterns that are built from the patterns recognized by the first layer, such as eyes, noses, and ears. The third layer may learn to recognize patterns that are built from the patterns recognized by the second layer, like faces. Now you can see that our neural network for recognizing the images was somewhat naive; we did not try to implement actual deep learning. By building abstractions on abstractions, we expect our network to find patterns that humans find, from structures in sentences, to signs of malignancy in medical images, to recognizing handwritten characters, to detecting online fraud.

然而，你可能会说，这一切都归结于在简单的构建块（人工神经元）上更新简单的值。你说得对。当人们意识到这一点时，有时会感到失望。他们想了解机器学习和深度学习是什么，而答案的简单性却令人失望：看似拥有人类能力的东西，竟然可以简化为一些基本操作。或许我们更愿意寻找更复杂的东西，这无疑会提升我们的自尊心。

Yet, you may say, it all boils down to updating simple values on simple building blocks—the artificial neurons. And you would be correct. When people realize that, sometimes they feel let down. They want to learn what machine and deep learning are, and the simplicity of the answer disappoints: something that appears to have human capabilities can be reduced to fundamentally elementary operations. Perhaps we would prefer to find something more involved, which would not fail to flatter our self-esteem.

然而，我们不应忘记，在科学领域，我们相信自然可以用第一原理来解释，并努力寻找尽可能简单的原理。这并不排除由简单规则和基本构件衍生出复杂的结构和行为。人工神经元比生物神经元简单得多，即使生物神经元的工作原理可以用简单的模型来解释，但正是由于大量相互连接的生物神经元，我们所知的智能才得以产生。

We should not forget, however, that in science we believe that nature can be explained from first principles, and try to find such principles that are as simple as possible. That does not preclude complex structures and behaviors arising out of simple rules and building blocks. Artificial neurons are much simpler than biological ones, and even if the workings of biological neurons can be explained in simple models, it is thanks to the vast number of interconnected biological neurons that intelligence, as we know it, can arise.

这有助于我们正确看待一些事情。诚然，人工神经网络的潜力巨大。然而，要使它们发挥作用，需要惊人的人类创造力和巨大的工程努力。我们在这里仅仅触及了皮毛。例如，以反向传播为例。这是神经网络背后的基本算法，使我们能够高效地执行本质上是寻找数学导数的过程。研究人员一直在忙于设计高效的计算技术，例如自动微分，这是一种已被广泛采用的计算导数的机制。或者以计算神经网络参数变化的具体方法为例。各种不同的优化器已经开发出来，使我们能够部署越来越大的网络，同时效率也越来越高。谈到底层硬件，硬件工程师正在设计更好的以及更先进的芯片，能够以更低的计算能力更快地运行更多神经计算。纵观网络架构，新的神经网络架构不断涌现，这些架构在现有架构的基础上得到了改进。这为研究和实验提供了温床，甚至包括构建用于设计其他神经网络的神经网络的努力。因此，每当你看到新闻报道某个神经网络取得了新的成就时，都要向那些辛勤工作的人们致敬，正是他们成就了这一切。9

This helps put some things into perspective. True, artificial neural networks can be uncanny in their potential. In order to make them work, however, an amazing amount of human creativity and terrific engineering effort is required. We have only scratched the surface in our account here. For instance, take backpropagation. That is the fundamental algorithm behind neural networks, allowing us to perform efficiently what is at heart a process of finding mathematical derivatives. Researchers have been busy devising efficient calculation techniques, such as automatic differentiation, a mechanism for calculating derivatives that has been widely adopted. Or take the exact way that changes in the neural network parameters are calculated. Various different optimizers have been developed, allowing us to deploy bigger and bigger networks that are at the same time more and more efficient. Turning to the underlying hardware, hardware engineers are designing better and better chips to run more and more neural computations faster while using less computing power. Looking at network architectures, new neural network architectures are proposed that improve on existing ones. This is a hotbed of research and experimentation, and even encompasses efforts to build neural networks that design other neural networks. So every time you see a news report that a neural network has reached a new achievement, doff your hat to the hardworking people who made this possible.9

前言

Preface

第一章

Chapter 1

第二章

Chapter 2

第三章

Chapter 3

第四章

Chapter 4

第五章

Chapter 5

第六章

Chapter 6

结语

Epilogue